Systems and methods for mitigating gesture input error

ABSTRACT

Devices, systems, and techniques for mitigating error from gesture input are disclosed. A system may receive an indication of a first gesture input, determine that the first gesture input is an indicator gesture input, and, responsive to the determination that the first gesture input is the indicator gesture input, enter a gesture control mode during which the system is configured to control one or more actions related to a medical procedure. Only during the gesture control mode, the system may receive an indication of a second gesture input associated with the medical procedure and, responsive to receiving the indication of the second gesture input, control, based on the second gesture input, at least a portion of the medical procedure. Additionally, or alternatively, the system may employ other error mitigation techniques for gesture input related to medical procedures.

This application is a continuation of U.S. patent application Ser. No.14/607,885, filed Jan. 28, 2015, the entire contents of which areincorporated herein by reference.

TECHNICAL FIELD

The disclosure relates to medical systems and, more particularly,medical system user interfaces.

BACKGROUND

A computing device (e.g., a workstation, surgical suite, programmingdevice, tablet computer, and notebook computer) typically utilizes oneor more input devices that convert user provided input into electricalsignals representing the user input and interpretable by the computingdevice. Example input devices may include keyboards, mice, joysticks,touch pads, dedicated hardware buttons, touch screens, microphones, andcameras. Through use of one or more of these input devices, a user mayinterface with one or more applications executing on the computingdevice. For example, the user may start or terminate an application,input data into the application, request the application to retrievedata stored in a data storage device, or command the application tocontrol a computer-controlled machine to perform one or more tasks.

SUMMARY

In general, devices, systems, and techniques for mitigating error fromgesture input in a medical system are described. A medical system maycontrol one or more actions related to a medical procedure based ongesture inputs received from a user, e.g., a clinician. One or moresensors, e.g., camera-based sensors or presence-sensitive sensors, maybe configured to detect hand motions or hand configurations presented bythe user as gesture input requesting that the system control an actionrelated to the medical procedure. The system may incorporate one or moremechanisms for mitigating error that may occur when detecting suchgesture input. For example, the system may require an indicator gestureinput or determination of a particular step in a medical procedure thattriggers entry into a gesture control mode during which the system canreceive gesture input to control one or more actions. In anotherexample, the system may terminate the gesture control mode responsive toa time-out period or completion of the step of the medical procedure. Insome examples, the system may limit the rate of change or adjustmentrange for adjustments to a medical procedure parameter made using agesture input. These and other error mitigation techniques describedherein may be used alone or in combination when receiving gesture input.

In one example, the disclosure is directed to a method includingreceiving, by a processor, an indication of a first gesture input,determining, by the processor, that the first gesture input is anindicator gesture input, responsive to the determination that the firstgesture input is the indicator gesture input, entering, by theprocessor, a gesture control mode during which the processor isconfigured to control one or more actions related to a medicalprocedure, receiving, by the processor and during the gesture controlmode, an indication of a second gesture input associated with themedical procedure, and responsive to receiving the indication of thesecond gesture input, controlling, by the processor and based on thesecond gesture input, at least a portion of the medical procedure.

In another example, the disclosure is directed to a system that includesone or more processors configured to receive an indication of a firstgesture input, determine that the first gesture input is an indicatorgesture input, responsive to the determination that the first gestureinput is the indicator gesture input, enter a gesture control modeduring which the processor is configured to control one or more actionsrelated to a medical procedure, receive, during the gesture controlmode, an indication of a second gesture input associated with themedical procedure, and responsive to receiving the indication of thesecond gesture input, control, based on the second gesture input, atleast a portion of the medical procedure.

In another example, the disclosure is directed to a computer-readablestorage medium including instructions that, when executed, cause one ormore processors to receive an indication of a first gesture input,determine that the first gesture input is an indicator gesture input,responsive to the determination that the first gesture input is theindicator gesture input, enter a gesture control mode during which theprocessor is configured to control one or more actions related to amedical procedure, receive, during the gesture control mode, anindication of a second gesture input associated with the medicalprocedure, and, responsive to receiving the indication of the secondgesture input, control, based on the second gesture input, at least aportion of the medical procedure.

In another example, the disclosure is directed to a method that includesreceiving, by a processor, an indication to enter a gesture control modeduring which the processor is configured to control one or more actionsrelated to a medical procedure, responsive to receiving the indication,entering, by the processor, the gesture control mode, monitoring, by theprocessor, field input data received during the gesture control mode forone or more control gesture inputs, the one or more control gestureinputs requesting the processor control the one or more actions relatedto the medical procedure, determining, by the processor and during thegesture control mode, to exit the gesture control mode, and responsiveto determining to exit the gesture control mode, exiting, by theprocessor, the gesture control mode, wherein exiting the gesture controlmode disables gesture input control of the one or more actions relatedto the medical procedure.

In another example, the disclosure is directed to a system that includesone or more processors configured to receive an indication to enter agesture control mode during which the processor is configured to controlone or more actions related to a medical procedure, responsive toreceiving the indication, enter the gesture control mode, monitor fieldinput data received during the gesture control mode for one or morecontrol gesture inputs, the one or more control gesture inputsrequesting the processor control the one or more actions related to themedical procedure, determine, during the gesture control mode, to exitthe gesture control mode, and, responsive to determining to exit thegesture control mode, exit the gesture control mode, wherein exiting thegesture control mode disables gesture input control of the one or moreactions related to the medical procedure.

In another example, the disclosure is directed to a computer-readablestorage medium including instructions that, when executed, cause one ormore processors to receive an indication to enter a gesture control modeduring which the processor is configured to control one or more actionsrelated to a medical procedure, responsive to receiving the indication,enter the gesture control mode, monitor field input data received duringthe gesture control mode for one or more control gesture inputs, the oneor more control gesture inputs requesting the processor control the oneor more actions related to the medical procedure, determine, during thegesture control mode, to exit the gesture control mode, and responsiveto determining to exit the gesture control mode, exit the gesturecontrol mode, wherein exiting the gesture control mode disables gestureinput control of the one or more actions related to the medicalprocedure.

The details of one or more example are set forth in the accompanyingdrawings and the description below. Other features, objects, andadvantages will be apparent from the description and drawings, and fromthe claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a conceptual diagram illustrating an example system thatincludes a gesture control system for controlling one or more actions ofa medical procedure based on received gesture input.

FIG. 2 is a conceptual diagram illustrating an example system thatincludes a gesture control device and implantable medical device thatdetect gesture input from a user for controlling one or more actions ofa medical procedure.

FIG. 3 is a block diagram of an example gesture control device of FIGS.1 and 2.

FIG. 4 is a block diagram of an example implantable medical device ofFIGS. 1 and 2.

FIG. 5 is a block diagram of an example external programmer of FIG. 2.

FIG. 6 is a block diagram of an example networked server.

FIG. 7 is a conceptual diagram illustrating an example system thatincludes the networked server of FIG. 6 for determining gesture inputbased on received field input data.

FIGS. 8A and 8B are conceptual diagrams of example gestures for anindication gesture input and a control gesture input.

FIGS. 9A and 9B are conceptual diagrams of example gestures for acontrol gesture input and a confirmation gesture input.

FIG. 10 is a conceptual diagram of example gesture that is authenticatedas provided by a particular user.

FIG. 11 is a flow diagram that illustrates an example process forentering a gesture control mode in response to receiving an indicatorgesture input.

FIG. 12 is a flow diagram that illustrates an example process forexiting a gesture control mode in response to a change in a medicalprocedure.

FIG. 13 is a flow diagram that illustrates an example process forreceiving a confirmation gesture input that confirms a control gestureinput.

FIG. 14 is a flow diagram that illustrates an example process forsuspending a gesture control mode in response to detecting an unexpectedobject within en envelope within which gesture input is detected.

FIG. 15 is a flow diagram that illustrates an example process forconfirming a gesture input is from an expected user.

FIG. 16 is a flow diagram that illustrates an example process fordetecting a gesture input based on data generated from an externalsensor and an implantable medical device.

FIG. 17 is a flow diagram that illustrates an example process forlimiting adjustments to parameters for gesture inputs.

DETAILED DESCRIPTION

This disclosure is generally directed to devices, systems, andtechniques for mitigating error that may occur due to gesture inputcontrol of medical procedures. Gesture interfaces, those user interfaceswhich are configured to receive gesture input from a user (e.g.,stationary hand positions, hand movements, or some combination thereof),provide flexible mechanisms for computer-human interaction. A systemutilizing gesture input may provide unique input mechanisms appropriatefor a variety of actions, reduce input complexity, and/or eliminate theneed for the user to physically touch an input device.

However, the detection of hand movements or hand positions in gestureinterfaces includes an inherent increase in probability of detectinginadvertent or unwanted gestures that result in the system performing anunintended action. In the context of consumer electronic devices, theseunintended actions may be of low consequence because correcting theunintended action may only require an additional navigation step throughthe user interface or providing another input that will “undo” orreverse the previous unintended input. In contrast, inadvertent gestureinput provided to a system that controls a medical procedure may resultin an unintended action of high consequence to a patient receiving themedical procedure.

As discussed herein, systems may be configured to receive and managegesture input to reduce or eliminate gesture input-related errors formedical procedures. A system may control one or more actions related toa medical procedure based on one or more gesture inputs received from auser, e.g., a clinician or a patient. One or more sensors, e.g.,camera-based sensors, capacitive sensors, resistor sensors orpresence-sensitive sensors, may be configured to detect hand motions orhand configurations presented by the user as gesture input requestingthat the system control an action related to the medical procedure. Thesystem may incorporate one or more mechanisms for mitigating error thatmay occur when detecting such gesture input. For example, a gesturecontrol system (e.g., one or more of a standalone computing device, anetworked server, and/or external programmer) may be configured to onlyreceive gesture input that controls an action of a medical procedureduring a gesture control mode. The system may initiate gesture controlmode in response to receiving an indication gesture input or determiningthat a predetermined step during the medical procedure as started. Thesystem may exit, or stop, the gesture control mode in response toreceiving a termination gesture input or determining that apredetermined step during the medical procedure has ended.

In addition, or alternative, to implementing a gesture control mode, asystem may be configured to ensure that a gesture input is received froma known user. For example, the system may be configured to detect anyunexpected object within an envelope within which the gesture input isdetected. The unexpected object may be the hand of a second user, anobject attached to the user's hand or finger, or any other abnormalitythat reduces the accuracy of gesture input detection from the user. Insome examples, changes in lighting or a change in the angle between agesture sensor and the user may manifest as the detection of anunexpected object. In response to detection of an unexpected object, thesystem may suspend a gesture control mode or otherwise lockout gestureinput control of any actions related to the medical procedure.

A system may also mitigate error with gesture input by limiting themagnitude of any adjustment that can be changed by gesture input. Thesystem may apply a factor to gesture input that limits a rate of changeor adjustment range for any action controlled by gesture input. Forexample, the factor may establish a small rate of change for anyadjustments made using gesture input such that a gesture input cannotunintentionally cause large or drastic actions during the medicalprocedure. The system may also set an adjustment range specific togesture input such that the system only changes a parameter value withina limited or small range in response to gesture input. In this manner,the system may require increased duration and/or magnitude of a gestureinput to adjust a parameter to the same degree as a smaller adjustmentvia a physical button or knob.

A gesture control system may employ one or any combination of techniquesto minimize or eliminate error that may occur when controlling actionsbased on gesture input. Employment of such error mitigation techniquesmay improve gesture interface technologies and systems related tocontrol of medical devices during medical procedures. These errormitigation techniques may also improve the functioning of controlsystems by reducing computations required to process gesture input whengesture input cannot be used for control or reducing the computationalresources needed to correct unintended user input that may occur withoutsuch error mitigation techniques. A system employing error mitigationtechniques described herein may also decrease user fatigue by reducingor eliminating unintended gesture input that requires correction by theuser.

FIG. 1 is a conceptual diagram illustrating example system 10 thatincludes a gesture control device 22 for controlling one or more actionsof a medical procedure based on received gesture input from user 14. Asshown in FIG. 1, system 10 may include gesture control device 22, outputdevices 24 and 26, camera-based sensors 30A and 30B (collectively“sensors 30”), operating table 20, and implantable medical device (IMD18). Patient 12 is shown as lying down on operating table 20 and withIMD 18 implanted within patient 12. IMD 18 may or may not have beenimplanted during the medical procedure example in system 10. IMD 18 maybe configured to deliver a therapy to patient 12. User 14 (e.g., aclinician or other healthcare professional) may provide gesturesdetectable by one or more of sensors 30 and interpretable by gesturecontrol device 22 as gesture input that controls one or more actionsrelated to a medical procedure being performed on patient 12. In someexamples, system 10 may also include an external programmer (not shown)configured to communicate with and program operations of IMD 18 and/orcommunicate with gesture control device 22.

A medical procedure may include any actions related to a medicalpatient, such as diagnosis and/or treatment of patient 12. Examplemedical procedures may also include manual examination by a physician,non-invasive data collection and/or imaging, removal of a fluid ortissue sample, surgical intervention, implantation of a medical device,adjustment or removal of a medical device from tissue, or adjustment ofone or more therapy parameters that at least partially define deliveryof therapy from a medical device or monitoring of a patient. In thismanner, the medical procedure may include actions provided by a medicalprofessional for the purpose of examining, diagnosing, or treating thepatient. Medical procedures may be any action performed by a medicalprofessional (e.g., a physician, clinician, or nurse) in relation to apatient or in association with one or more computing devices or medicaldevices associated with the diagnosis or treatment of the patient.

System 10 may be configured to capture, or detect, gestures performed byuser 14 during a medical procedure. In one example, user 14 may use oneor both of hands 16A and 16B to make gestures detectable by one or moresensors 30. Gestures may include specific locations within a detectableenvelope, stationary configurations and/or movements of one or morefingers or hands. For example, gestures may include a predeterminednumber of extended fingers, a shape of one or more fingers or hands, aspatial relationship of one or more fingers to another one or morefingers, a movement of one or more fingers or hands to another one ormore fingers or hands, or any combination thereof. Although gestureswill be generally described with relation to fingers and/or hands, agesture may involve one or more other portions of the anatomy of user14. In some examples, gestures may be performed by multiple users.

Sensors 30A and 30B may be configured to detect gestures performed byuser 14 and generate a signal representative of the detected gestures.Sensors 30A and 30B may be camera-based sensors that obtain lightreflected off of user 14. Sensors 30 are shown as mounted onto stand 28to position sensors 30 in front of user 14, but sensors 30 may beattached to any structure or otherwise positioned to detect the gesturesof user 14. For examples, sensor 30A and/or sensor 30B may include oneor more sensors (e.g., charge-coupled device (CCD) or complementarymetal-oxide-semiconductor (CMOS)) configured to convert visible light toelectrical signals representing the visible light. In other examples,sensors 30 may include other sensors that are configured to captureinfra-red electromagnetic radiation and/or any other mediumrepresentative of the position or movement of user 14. In some examples,sensors 30 may include electrical field sensors, heat sensors, or anysensors configured to detect the position and/or movement of user 14within a location envelope detectable by the sensors. Sensors 30 may, insome examples, include capacitive or resistive sensors associated with atouchscreen device.

As shown in the example of FIG. 1, system 10 may include two sensors 30to generate stereo images (or multiple vantage points) of user 14 thatincludes depth of field instead of single plane information typicallygenerated from a single sensor. In this manner, sensors 30 may beconfigured to detect hands 16A and 16B in three dimensions and locatewhere hands 16 are located within the space of the room in which sensors30 are located. In some examples, gesture control device 22 may use thisspatial information of hands 16 within the room to determine whichgesture input has been provided. For example, gesture control device 22may determine a particular gesture input is associated with control of afirst device when the gesture input is detected in a particular cornerof the room as opposed to the particular gesture input being associatedwith control of a second device when the gesture input is detectedadjacent to patient 12. In this manner, gesture control device 22 mayuse identified items around the room and in relation to hands 16 todetermine the functionality to which the gesture input should be applied(e.g., hands 16 within a spatial envelope of a device control thatdevice or hands 16 within a special envelope of patient 12 controlparameters of IMD 18). Using the spatial location of gestures within thedetection area of sensors 30 may provide a larger vocabulary of gestureinputs. In other words, the same gesture could be combined with thespatial data of where the gesture was detected in order to generatemultiple different gesture inputs (e.g., the gesture input is based onthe gesture and the spatial location of the gesture within the room).

In some examples, three or more sensors may be used to detect gesturesprovided by user 14. Multiple sensors (e.g., sensors 30A and 30B) may beof the same type or sensing modality (e.g., all optical sensors or allultrasonic sensors) or of different types or sensing modalities (e.g.,an optical sensor, an infrared sensor, and an ultrasonic sensor) tominimize error conditions that may not affect all of the different typesof sensors. The sensing modality may be specific to the physical mediumthrough which the sensor detects the gestures (e.g., frequencies oflight, heat, vibrations, sound or pressure waves, electrical fields,etc.). In other words, system 10 may provide more robust detection ofgestures by detecting the gestures using multiple types of detectionmodalities. If one of the modalities is compromised, such as lightingchanges for an optical sensor or electronic interference for anothertype of sensor, the one or more additional modalities may remainoperational and sufficient to detect the gestures provided by user 14.

In other examples, a single sensor may be configured to detect gesturesinstead of multiple sensors. Sensors configured to detect gestures maybe placed at a variety of locations within an operating room,examination room, or any other environment within which gestures are tobe detected. For example, system 10 may include one or more sensorsabove user 14 and patient 12 (e.g., sensors attached to a ceiling mountor overhead mount), below user 14 and patient 12 (e.g., sensors withinor on the floor and/or operating table 20), attached to gesture controldevice 22, or even attached to patient 12 and/or user 14. In someexamples, user 14 may wear one or more devices detectable by sensors 30to facilitate or improve the detection of user gestures.

Sensors 30 may generate one or more signals representative of thedetected gestures, and the signals may be referred to as field inputdata. Sensors 30 may transmit the field input data to one or moreprocessors of gesture control device 22 via a wired and/or wirelessconnection. Gesture control device 22 may receive the field input data,determine when one or more detected gestures result in a respectivegesture input, and determine what actions to take in response to thegesture input. Gesture control device 22 may compare the gesture inputto predetermined gesture inputs stored in memory, or interface with anetworked computing device that performs the comparison, to determinethe request of each gesture input. In addition, gesture control device22 may employ one or more of the error mitigation techniques describedherein in order limit gesture inputs to those gesture inputs actuallyintended to be detected from user 14.

Gesture control device 22 may be configured to output informationrelated to gesture inputs to user 14. Gesture control device 22 may bein communication with display 24 (e.g., an output device) and speaker 26(e.g., another output device). Gesture control device 22 may controldisplay 24 and/or speaker 26 to deliver information to user 14 regardingthe gesture input and/or actions controlled by the gesture input for themedical procedure. Display 24 may display, based on output and/orinstructions generated by gesture control device 22, visual informationsuch as the most recent gestures detected, the resulting gesture input,and/or what actions will be controlled based on the gesture input.

In addition, display 24 may be configured to present informationregarding the interaction mode of gesture control device 22. Forexample, display 24 may display an indication of the type of gestureinteraction mode currently used by gesture control device 22. Theindication may indicate whether gesture control is active (or “on” toindicate that the system is currently detecting gestures or usinggesture input) or inactive (or “off” to indicate that the system is notcurrently detecting gestures or using gesture input). In some examples,the indication may alternatively, or additionally, indicate the type ofgesture interaction mode used to detect gestures from user 14. Forexample, the type of interaction mode indicated may be representative ofthe modality used to detect gestures (e.g., optical, infrared, orultrasonic) and/or the sensors used to detect gestures. The indicationof the gesture interaction mode may include one or more pictures orsymbols (e.g., a graphical icon), words, characters, numericalrepresentations, colors, shading variations for respective modes, or anyother visual representation of the type of gesture interaction modecurrently used by gesture control device 22.

Speaker 26 may deliver audio information regarding the gesture input inaddition to, or alternative to, display 24. User 14 may utilizeinformation delivered by gesture control device 22 via display 24 and/orspeaker 26 to ensure that gesture input is correct. Gesture controldevice 22 may also output, via display 24 and/or speaker 26, anindication of when the gesture control mode has been entered and/or whatstep is currently active for the medical procedure.

Gesture control device 22 may interface with one or more medical devicesconfigured to perform an action related to a medical procedure forpatient 12. In this manner, gesture control device 22 may control, basedon received gesture input, the one or more medical devices. Gesturecontrol device 22 may be in communication with the medical devices viawired and/or wireless communication protocols. For example, thecontrolled medical device may be a surgical device configured to operateon patient 12 (e.g., a device configured to cut, pierce, ablate, remove,attach, or mend patient tissue). In one example, gesture control device22 may be configured to control the surgical device to deliver ablationenergy to patient 12 in response to receiving the corresponding controlgesture input. In other examples, the controlled medical device may be asurgical device configured to implant, or assist with the implantationof, one or more additional medical devices. In one example, gesturecontrol device 22 may be configured to control the position of acatheter insertion device based on received gesture input. In thismanner, gesture control device 22 may be configured to control of asurgical device to operate with respect to patient 14 receiving themedical procedure. In some examples, the controlled medical device maybe an external or implantable medical device configured to remain withpatient 12 outside of the clinic and provide therapy to and/or monitorphysiological status of, patient 12. In this manner, gesture controldevice 22 may be configured to adjust, based on the received gestureinput, a parameter value of one or more parameters that at leastpartially defines a medical therapy.

A medical procedure that may be at least partially controlled by gestureinput includes surgical procedures, such as the removal or repair ofpatient tissue or implantation of one or more medical devices. In otherexamples, a medical procedure that may be at least partially controlledby gesture input may include the adjustment of one or more parametersthat at least partially define a therapy delivered by a medical device.For example, the therapy may be an electrical stimulation therapydelivered by an implantable medical device. Alternatively, the therapymay be a drug delivery therapy delivered by an implantable medicaldevice.

As described herein, system 10 may be configured to employ one or moreerror mitigation techniques for gesture input interfaces. For example,gesture control device 22 may be configured to receive an indication ofa first gesture input and determine that the first gesture input is anindicator gesture input. Responsive to the determination that the firstgesture input is the indicator gesture input, gesture control device 22may be configured to enter a gesture control mode during which theprocessor is configured to control one or more actions related to amedical procedure and receive, during the gesture control mode, anindication of a second gesture input associated with the medicalprocedure. Gesture control device 22 may then, responsive to receivingthe indication of the second gesture input, control, based on the secondgesture input, at least a portion of the medical procedure.

Gesture control device 22 may be configured to also output, for displayvia a user interface, an indication that the gesture control mode isentered. For example, gesture control device 22 may control display 24and/or speaker 26 to present a visual and/or audible indication to user14 that the gesture control mode has been entered and gesture input canbe used for control of the medical procedure. One or more ofcamera-based sensors 30 may be configured to sense the first and secondinputs from user 14 and transmit the gesture input to gesture controldevice 22 as field input data. In some examples, gesture control device22 may determine from the field input data whether or not a gestureinput has occurred. Alternatively, gesture control device 22 may receivean indication of what gesture input was detected and then determine whataction to take for the corresponding gesture input that was received.

The first and second gesture input received by gesture control device 22may be indicative of certain types of gesture input. For example, thesecond gesture input that is received during the gesture control modemay be one of a plurality of pre-defined control gestures that aredetectable. In other words, the control gestures may be stored in alibrary of control gestures such that gesture control device 22 candetermine when a control gesture has been provided by user 14. The firstgesture input may be described as an indicator gesture input requestingentry into the gesture control mode so that control gesture input can beutilized by gesture control device 22. In some examples, the indicatorgesture input may be different from any of the possible control gestureinput such that the gesture input controlling the gesture control modestatus (e.g., whether the gesture control mode has been entered or not)remains separate from any gesture input used to control the one or moreactions related to the medical procedure. This separation may, in someexamples, reduce the possibility that the gesture control mode isinadvertently entered. Gesture control device 22, or another computingdevice that stores and/or manages the library of control gestures, maybe configured to create, update, or otherwise manage the controlgestures stored in the library in response to user input requesting achange to the library. In this manner, the library may be configurable,selectable, and/or created according to input provided by one or moreusers. In this manner, the library may be configured to be specific forone or more specific purposes and/or one or more specific users. Thelibrary may also include indicator gestures or any other types ofgestures in addition to, or alternative to, control gestures.

In some examples, the indictor gesture input may be a low-error gestureinput that is easily detected by gesture control device 22. Examplelow-error gesture inputs may include those gestures with multipleidentifiable portions of the anatomy of user 14 (e.g., five outstretchedfingers of one hand or one hand positioned in relation to anotherportion of the user anatomy such as the nose or ear). In contrast,control gesture inputs may not be low-error gesture input because thecontrol gesture input may include motion or some other complex handconfiguration needed to indicate a desired change in the medicalprocedure. Gesture inputs that are not specifically classified as“low-error” gestures herein are not necessarily prone to high error orotherwise compromise the accuracy of the gesture input or thecorresponding action requested by the gesture input. For example, somegestures used for control gesture input (e.g., moving two fingerstogether to indicate reducing a parameter value) may involve determiningchanges to a hand in space, but such a gesture may still providesufficient specificity for a control gesture input. However, low-errorgesture inputs may be particularly resistant to error in the user makingthe gesture and/or gesture control device 22 detecting the gesture. Asdescribed herein, confirmation gesture input and/or termination gestureinput may also be selected with low-error gestures in some examples.Control gesture input may be low-error gestures in some examples, butthe control scheme may only allow pre-selected changes or smallincremental changes in the adjusted control of the corresponding actionof the medical procedure.

In some examples, gesture control device 22, or another computing deviceassociated with system 10, may be configured to classify gestures aslow-error gestures, high-error gestures, or provide a scale indicativeof the relative ease at which gesture control device 22 can accuratelyidentify the gesture from a user. The computing device may thus analyzea particular gesture for qualitative and/or quantitative measure of theerror rates for particular gestures. This measure may be a scaleindicating the relative error rates for each of a plurality of gestures.The relative error rate for a given gesture may be retrieved from astored databased of predetermined error rates for types of gesturesand/or calculated based on the gesture selected for use. For example,the gesture control device 22 may recognize attributes of the gesture(e.g., stationary, motion, range of motion, speed of motion, number offingers, number of hands, duration of gesture, steps in each gesture,size of detectable elements in the gesture, etc.) and assign an expectederror value to each of these attributes. For example, gesture controldevice 22 may apply a higher expected error value to motion attributesthan stationary attributes. Gesture control device 22 may then calculatean error rate for the gesture based on the error values of eachattribute (e.g., a sum of the error values or weighted average of theerror values).

Based on the error rate for a gesture, gesture control device 22 maycontrol a display to present an indication of the error rate for thegesture. The indication may be the actual error rate value or aclassification of the error rate within two or more groups (e.g., alow-error gesture and a high-error gesture) or along a scale of three ormore levels of error. The user can use the classification or error rateto determine to which functionality the gesture can be used for controlor indicator gestures. In some examples, gesture control device 22 maycontrol the display to present an error or confirmation message if theerror rate or classification of the gesture is not suitable for theselected functionality requested to be controlled by the gesture.Gesture control device 22 may request confirmation input (or even rejectthe assignment of a gesture to a user selected function) for high-errorgestures selected to control high risk activities associated withpatient 12. For example, gesture control device 22 may prompt user 14 toconfirm that a high-error gesture of a complex hand motion gestureshould be assigned to control the initiation and termination of ablationenergy. In other words, such high risk activity may be more suitablycontrolled by a low-error gesture such as a stationary hand with fiveoutstretched fingers. If gesture control device 22 rejects theassignment of a gesture to a particular function or activity, gesturecontrol device 22 may present an error message to user 14 indicating therejection of the gesture assignment and, in some examples, anexplanation for why the gesture was not assigned to the function.

Gesture control device 22 may exit the gesture control mode based on oneor more exit triggers. The exit triggers may be provided to prevent thegesture control mode from remaining active for periods of time duringwhich gesture input may be inadvertent (e.g., during portions of themedical procedure in which gesture inputs should not be used for controlor after the period of time a user requires gesture input for control ofthe medical procedure). In other words, limiting the period of timeduring which gesture input can control one or more actions of a medicalprocedure may limit errors related to gesture input such as inadvertentgesture inputs.

In one example, gesture control device 22 may be configured to track aduration of time from when the gesture control mode was entered, comparethe duration of time to a predetermined gesture control timeout, anddetermine that the duration of time exceeds the predetermined gesturecontrol timeout. Responsive to determining that the duration of timeexceeds the predetermined gesture control timeout, gesture controldevice 22 may exit the gesture control mode. In this manner, thepredetermined gesture control timeout may define the predeterminedperiod of time during which a gesture control mode may remain active.For example, the predetermined gesture control timeout may be betweenapproximately 10 seconds and 2 hours. In another example, thepredetermined gesture control timeout may be between 1 minute and 5minutes. The gesture control timeout may be selected by gesture controldevice 22 based on a specific medical procedure and/or a particular stepwithin a medical procedure such that the duration of the gesture controltimeout is matched to the length of time the user needs the gesturecontrol mode to be active to perform a particular action of the medicalprocedure and/or the entire medical procedure. In some examples, gesturecontrol device 22 may be configured to receive a user input (e.g., amechanical or gesture input) requesting an extension to thepredetermined gesture control timeout. Gesture control device 22 maygrant the request to the extension in all cases or only in response todetermining that the medical procedure, or step of the medicalprocedure, allows for extensions to the gesture control timeout.

In another example, gesture control device 22 may exit the gesturecontrol mode based on the progress of the medical procedure. Gesturecontrol device 22 may be configured to monitor progress of the medicalprocedure, where the medical procedure includes a plurality of steps. Atleast one of the plurality of steps may be configured to accept gestureinput for control of an action corresponding to the step. In someexamples, initiation gesture input and/or control gesture input may bereceived during the duration of the step of the medical procedure.Gesture control device 22 may determine that the one step of the medicalprocedure has been completed (e.g., based on an input received thatdefines the completion of the step or identifying that a subsequent stepfor the medical procedure has started). In some examples, gesturecontrol device 22 may determine that a step of the medical procedure hasbeen completed in response to detecting or identifying use of otherdevices or instruments (e.g., a surgical instrument or other computingdevice). In one example, gesture control device 22 may determine thatthe surgical planning step has been completed in response to detectingthat a surgical navigation system and/or user interface has beenactivated. Responsive to determining that the one step has beencompleted, gesture control device 22 may exit the gesture control mode.In some examples, the gesture control mode may span two or more steps ofthe medical procedure such that gesture control device 22 enters thegesture control mode during one step of the medical procedure and exitsthe gesture control mode after completion of a different and subsequentstep of the medical procedure. In this manner, the gesture control modemay only be active, or entered, during steps of the medical procedureappropriate for gesture input control of one or more actions.

Alternative, or in addition, to other exit triggers, gesture controldevice 22 may be configured to exit or suspend the gesture control modein response to detecting unexpected objects (e.g., objects, anomalies,motions, etc.) in an envelope within which gestures from user 14 can bedetected by sensors 30, for example. In one example, gesture controldevice 22 may receive field input data from sensors 30 during thegesture control mode (e.g., when the gesture control mode is entered oractive). Gesture control device 22 may detect, from the field inputdata, an unexpected object in the envelope within which a previousgesture input was sensed or a gesture input is otherwise detectable. Theunexpected object may be a third hand or other anatomical structure(which may indicate a second user has entered the envelope), a device(e.g., a medical device such as a surgical instrument) in or attached toone of hands 16 of user 14, an object that obstructs the view of atleast a portion of the envelope from one or more or sensors 30, or anyother anomaly that may increase the probability of an inadvertentgesture input and/or limit the ability of gesture control device 22 toaccurately detect gesture input. Example anomalies detected by gesturecontrol device 22 may include the detection of unexpected gestures(e.g., gestures identifiably by gesture control device 22 but notassociated with the particular mode in which gesture control device 22is operating) or any other loss of system confidence in positive gesturedetection. Such loss of confidence may be the result of the detection ofmultiple different gestures at the same time (e.g., inability todistinguish between gestures) or a detected error with one or moresensors that detect gestures.

Responsive to detecting the unexpected object, gesture control device 22may be configured to exit, or suspend, the gesture control mode toprevent detection of erroneous gesture input due to the unexpectedobject. Once the gesture control mode is exited, gesture control device22 may require an indicator gesture input or another user input thatrequests re-entry of gesture control mode before gesture control device22 will re-enter the gesture control mode. Alternatively, gesturecontrol device 22 may automatically unsuspend, or re-enter, a gesturecontrol mode after receiving an indication to unsuspend the gesturecontrol mode. For example, gesture control device 22 may monitor thefield input data for changes to the unexpected object and unsuspend, orre-enter, the gesture control mode in response to determining that theunexpected object is no longer present in the envelope. In someexamples, gesture control device 22 may re-enter the gesture controlmode after a predetermined gesture control timeout has lapsed and, if anunexpected object remains in the envelope, the gesture control device 22may again suspend the gesture control mode.

In some examples, gesture control device 22 may exit the gesture controlmode in response to receiving user input requesting that the gesturecontrol mode be terminated. For example, gesture control device 22 maybe configured to receive field input data or gesture input and determinethat the third gesture input is a termination gesture input (e.g., agesture input that requests termination of the gesture control mode).Responsive to determining that the gesture input is a terminationgesture input, gesture control device 22 may exit the gesture controlmode. In some examples, the termination gesture input may include agesture different than a control gesture and an indicator gesture. Inother words, the termination gesture input may be specific totermination of the gesture control mode.

As discussed herein, gesture control device 22 may be controlled toexit, or terminate, the gesture control mode used to enable controlgesture input control of one or more actions related to the medicalprocedure. Gesture control device 22 may be configured to receive anindication to enter the gesture control mode during which gesturecontrol device 22 is configured to control one or more actions relatedto the medical procedure. In indication may be an indicator gestureinput, another user input, entering a new step in a medical procedure,or some other request to enter the gesture control mode. Responsive toreceiving the indication, gesture control device 22 enters the gesturecontrol mode.

During the gesture control mode, gesture control device 22 may monitorfield input data received during the gesture control mode for one ormore control gesture inputs. The one or more control gesture inputs maybe a request for gesture control device 22 to control one or moreactions related to the medical procedure, such as operating a surgicaldevice or adjusting a parameter value that at least partially definestherapy delivered by an implantable medical device. Gesture controldevice 22 may determine, during the gesture control mode, to exit thegesture control mode and, responsive to determining to exit the gesturecontrol mode, gesture control device 22 may exit the gesture controlmode. Exiting, or terminating, the gesture control mode may disablegesture input control of the one or more actions related to the medicalprocedure. In other words, gesture control device 22 may be configuredto accept gesture input control of an action of the medical procedureonly during the gesture control mode to mitigate error from inadvertentgesture input, for example.

Gesture control device 22 may be configured to exit the gesture controlmode based on the progress of the medical device. For example, gesturecontrol device 22 may be configured to monitor a progress of the medicalprocedure, where the medical procedure includes a plurality of stepsand, for one step of the plurality of steps, gesture control device 22is configured to accept gesture input controlling one or more actionsrelated to the medical procedure during the one step. Gesture controldevice 22 may determine, based on monitoring the medical procedureprogress, that the one step of the medical procedure has been completed,and, responsive to determining that the one step has been completed,gesture control device 22 may exit the gesture control mode. Forexample, gesture control device 22 may monitor the medical procedureprogress via user input identifying when the procedure moves to the nextstep, user input completing a step, and/or actions performed by one ormore devices in communication with gesture control device 22.

Gesture control device 22 may also monitor the progress of the medicalprocedure for multiple steps for which gesture input can be used tocontrol an action. Gesture control device 22 may accept one or moregesture inputs of a first plurality of gesture inputs during the firststep and determine that the medical procedure has entered a second stepof the medical procedure. The first and second steps may not besequential steps of the medical procedure. Responsive to thedetermination that the medical procedure has entered the second step ofthe medical procedure, gesture control device 22 may enter the gesturecontrol mode and accept one or more gesture inputs of a second pluralityof gesture inputs during the second step. The second plurality ofgesture inputs may be different than the first plurality of gestureinputs such that different sets of gesture inputs are used to controlactions in the respective steps. Gesture control device 22 may determinethat the second step of the medical procedure has been completed and,responsive to determining that the second step has been completed, exitthe gesture control mode for the second step.

In other examples, gesture control device 22 may use other techniquesfor determining when to exit the gesture control mode. For example,gesture control device 22 may exit the gesture control mode after apredetermined gesture control timeout has lapsed from when the gesturecontrol mode started. Predetermined gesture control timeouts may bedifferent and specific to respective steps within the medical procedure.For example, gesture control device 22 may track a first gesture controltimeout specific to a first step of the medical procedure and a secondgesture control timeout specific to a second step at a different portionof the medical procedure. The predetermined specific gesture controltimeouts may be pre-selected for a particular step of the medicalprocedure such that the gesture control mode is active only for the timenecessary to control the actions associated with the respective step ofthe medical procedure.

In another example, and as described herein, gesture control device 22may suspend or terminate the gesture control mode in response todetecting an unexpected object within the envelope for detecting gestureinput. For example, gesture control device 22 may receive first fieldinput data during the gesture control mode, detect, from the first fieldinput data, an unexpected object in an envelope within which gestureinput can be sensed, and, responsive to detecting the unexpected object,suspend the gesture control mode to prevent detection of erroneousgesture input due to the unexpected object. Gesture control device 22may also receive second field input data during the suspension of thegesture control mode, detect, from the second field input data, that theunexpected object is no longer in the envelope, and, responsive todetecting that the unexpected object is no longer in the envelope,re-enter the gesture control mode. Alternatively, gesture control device22 may simply terminate the gesture control mode in response todetecting the unexpected object.

Gesture control device 22 may also receive a confirmation gesture inputto confirm that a control gesture input was to be received by thegesture control device. For example, gesture control device 22 may beconfigured to receive, during the gesture control mode, an indication ofone or more control gesture inputs requesting gesture control device 22to control one or more actions related to the medical procedure. Gesturecontrol device 22 may also receive, during the gesture control mode, aconfirmation gesture input confirming that the one or more controlgesture inputs were to be detected. Responsive to receiving both the oneor more control gesture inputs and the confirmation gesture confirmationinput, gesture control device 22 may control, based on the one or morecontrol gesture inputs, the one or more actions related to the medicalprocedure. In this manner, the control gesture input may be used tominimize actions based on inadvertent gesture control inputs because aconfirmation gesture input may be required. In some examples, theconfirmation gesture input may be different from any control gestureinputs gesture control device 22 may use to control actions related tothe medical procedure. Confirmation gesture input may generally be asdistinct from control gesture inputs as possible (e.g., use of adifferent number of hands, stationary vs. motion gestures, or otherdissimilar gesture inputs). In some examples, the confirmation gestureinput may require a specific combination of other gestures to minimizethe possibility that a confirmation gesture input is inadvertentlydetected.

Gesture control device 22 may be configured to receive and detectdifferent gesture inputs in different orders. For example, gesturecontrol device 22 may require that an indicator gesture input forentering the gesture control mode is received prior to any controlgesture input or that control gesture input is received prior to anyconfirmation gesture input. Alternatively, gesture control device 22 maybe configured to receive different gesture inputs simultaneously. Forexample, gesture control device 22 may be configured to detect both anindicator gesture input for entering the gesture control mode at thesame time as a control gesture input and, subsequent to receiving bothinputs, control an action associated with the medical procedure.Similarly, gesture control device 22 may be configured to detect acontrol gesture input simultaneously with a confirmation gesture input.For example, user 14 may make a control gesture with hand 16A at thesame time that user 14 makes a confirmation gesture with hand 16B toconfirm that the control gesture is intended.

Although gesture control device 22 is described as an external devicethat houses one or more processors that determine the gesture input andcontrol one or more action associated with the medical procedure basedon the gesture input, one or more other devices may perform the featuresof gesture control device 22. For example, a networked server, a medicaldevice programmer, or an implantable medical device (e.g., IMD 18) maybe configured to house the one or more processors configured to performat least some of the features attributed to gesture control device 22.

As described herein, gesture input is generally described as requests toadjust a therapy, control a surgical process, or otherwise affect atreatment of patient 12. In other examples, gesture input may also beused to control a monitoring and/or diagnostic process for patient 12.In some examples, gesture control device 22 may be configured to controlthe flow of information between medical devices or from one medicaldevice to another device. For example, gesture control device 22 may beconfigured to move patient data from a diagnostic device (e.g., aheart-rate monitor or fluoroscope) to a networked device that managesthe electronic medical record (EMR) of patient 12. In another example,gesture control device 22 may be configured to associate a multiplemedical devices (e.g., connect a fluoroscope, computed-tomographyscanner, and a patient programmer that controls IMD 18) in response todetecting the appropriate gesture input from user 14. This associationof multiple medical devices using gesture input may allow user 14 tooptimize therapy parameter selection based on the output of one or moreof the associated medical devices. In this manner, gesture control ofmultiple devices and the flow of patient data immediately duringinteraction with patient 12 may improve patient safety, proceduraloutcomes, and codification of patient treatment.

FIG. 2 is a conceptual diagram illustrating example system 40 thatincludes gesture control device 22 and IMD 18 that detect gesture inputfrom a user for controlling one or more actions of a medical procedure.System 40 may be similar to system 10 described in FIG. 1. For example,systems 10 and 40 include sensors 30A and 30B in communication withgesture control device 22, patient 12, and IMD 18. However, system 40may incorporate detection capabilities from IMD 18 when determining agesture control input and/or confirming that the received gesturecontrol input is from the intended user (e.g., a clinician working withpatient 12). IMD 18 may be configured to deliver spinal cord stimulation(SCS) to a patient 12. In other examples, IMD 18 may additionally oralternatively be configured to provide peripheral nerve fieldstimulation (PNFS), occipital nerve stimulation, sacral nervestimulation (SNS), pelvic floor stimulation, or any other electricalstimulation therapy. Alternatively, or additionally, IMD 18 may beconfigured to deliver a different therapy (e.g., drug therapy), monitorone or more physiological parameters of patient 12, or provide any otherservice for patient 12.

As shown in FIG. 2, system 40 includes an IMD 18 and external programmer50 shown in conjunction with a patient 12, who is ordinarily a humanpatient. In the example of FIG. 2, IMD 18 is an implantable electricalstimulator that delivers neurostimulation therapy to patient 12, e.g.,for relief of chronic pain or other symptoms such as abnormal movements.Generally IMD 18 may be a chronic electrical stimulator that remainsimplanted within patient 12 for weeks, months, or even years. In theexample of FIG. 2, IMD 18 and leads 46A and 46B (collectively “leads46”) may be directed to delivering SCS therapy. In other examples, IMD18 may be a temporary, or trial, stimulator used to screen or evaluatethe efficacy of electrical stimulation for chronic therapy. IMD 18 maybe implanted in a subcutaneous tissue pocket, within one or more layersof muscle, or other internal location. IMD 18 may be coupled to leads46, but fewer or greater numbers of leads may be used in other examples.

Electrical stimulation energy, which may be constant current or constantvoltage based pulses, for example, is delivered from IMD 18 to one ormore targeted locations within patient 12 via one or more electrodes(not shown) carried by each of respective leads 46. The parameters for aprogram that controls delivery of stimulation energy by IMD 18 mayinclude information identifying which electrodes have been selected fordelivery of stimulation according to a stimulation program, thecombination of the selected electrodes, and the polarities of theselected electrodes, i.e., the electrode configuration for the program,and voltage or current amplitude, pulse frequency (or pulse rate), pulseshape, and pulse width of stimulation delivered by the electrodes.Electrical stimulation may be delivered in the form of stimulationpulses or continuous waveforms, for example.

In the example of FIG. 2, leads 46 are implanted and disposed withinpatient 12. Leads 46 are tunneled through tissue of patient 12 fromalong spinal cord 42 to a subcutaneous tissue pocket or other internallocation where IMD 18 is disposed (e.g., within the pocket exposed byskin opening 48. Although leads 46 may each be a continuous lead, one orboth of leads 46 may include a lead extension or other segments that mayaid in implantation or positioning of leads 46. In addition, proximalends of leads 46 may include a connector (not shown) that electricallycouples to a header of IMD 18. In addition to leads 46, additional leadsmay be implanted and coupled to IMD 18 or another implantable medicaldevice. In some examples, one or more leadless IMDs may be implantedwithin patient 12 and in communication with IMD 18 and/or externalprogrammer 50.

Leads 46 may each carry one or more electrodes that are placed adjacentto the target tissue, e.g., spinal cord 42 for spinal cord stimulation(SCS) therapy. One or more electrodes may be disposed at or near adistal tip of leads 46 and/or at other positions at intermediate pointsalong leads 46, for example. Electrodes of leads 46 are configured totransfer electrical stimulation generated by an electrical stimulationgenerator (or multiple generators) in IMD 18 to tissue of patient 12.The electrodes may be electrode pads on a paddle lead, circular (e.g.,ring) electrodes surrounding the body of the lead, conformableelectrodes, cuff electrodes, segmented electrodes (e.g., electrodesdisposed at different circumferential positions around the lead insteadof a continuous ring electrode), or any other type of electrodes capableof forming unipolar, bipolar or multipolar electrode combinations fortherapy.

IMD 18 may be configured to deliver electrical stimulation therapy topatient 12 via selected combinations of electrodes carried by one ormore of leads 46. The target tissue for the electrical stimulationtherapy may be any tissue affected by electrical stimulation energy,which may be in the form of electrical stimulation pulses or waveforms.In some examples, the target tissue includes nerves, smooth muscle, andskeletal muscle. In the example illustrated by FIG. 2, the target tissuefor electrical stimulation delivered via leads 46 is tissue proximatespinal cord 42 (e.g., one or more target locations of the dorsal columnsor one or more dorsal roots that branch from spinal cord 42). Leads 46may be introduced into spinal cord 42 via any suitable region, such asthe thoracic, cervical or lumbar regions. Stimulation of dorsal columns,dorsal roots, and/or peripheral nerves (e.g., afferent nerves) may, forexample, prevent pain signals from traveling through spinal cord 42 andto the brain of the patient. Patient 12 may perceive the interruption ofpain signals as a reduction in pain and, therefore, efficacious therapyresults. For treatment of other disorders, leads 46 may be introduced atany exterior location of patient 12.

Although leads 46 is described as generally delivering or transmittingelectrical stimulation signals, lead 46 may additionally transmitelectrical signals from patient 12 to IMD 18 for monitoring. Forexample, IMD 18 may utilize detected nerve impulses to diagnose thecondition of patient 12 or adjust the delivered stimulation therapy.Leads 46 may thus transmit electrical signals to and from patient 12.One or more of leads 46 may carry other sensors such as accelerometers,ultrasound sensors or transducers, and temperature sensors.

A user, such as a clinician or patient 12, may interact with a userinterface of an external programmer 50 to program IMD 18, duringimplantation of IMD 18 and/or after implantation during use of IMD 18.Programming of IMD 18 may refer generally to the generation and transferof commands, programs, or other information to control the operation ofIMD 18. In this manner, IMD 18 may receive the transferred commands andprograms from programmer 50 to control stimulation therapy. For example,external programmer 50 may transmit programs, parameter adjustments,program selections, group selections, user input, or other informationto control the operation of IMD 18, e.g., by wireless telemetry or wiredconnection. As described herein, information may be transmitted betweenexternal programmer 50 and IMD 18. Therefore, IMD 18 and programmer 50may communicate via wireless communication using any techniques known inthe art. Examples of communication techniques may include, for example,radiofrequency (RF) telemetry and inductive coupling, but othertechniques are also contemplated. In some examples, programmer 50 mayinclude a communication head that may be placed proximate to thepatient's body near the IMD 18 implant site in order to improve thequality or security of communication between IMD 18 and programmer 50.Communication between programmer 50 and IMD 18 may occur during powertransmission or separate from power transmission.

External programmer 50 wirelessly communicates with IMD 18 as needed toprovide or retrieve therapy information. Programmer 50 is an externalcomputing device that the user, e.g., a clinician and/or patient 12, mayuse to communicate with IMD 18. For example, programmer 50 may be aclinician programmer that the clinician uses to communicate with IMD 18and program one or more therapy programs for IMD 18. Alternatively,programmer 50 may be a patient programmer that allows patient 12 toselect programs and/or view and modify therapy parameters. The clinicianprogrammer may include more programming features than the patientprogrammer. In other words, more complex or sensitive tasks may only beallowed by the clinician programmer to prevent an untrained patient frommaking undesirable changes to IMD 18.

When programmer 50 is configured for use by the clinician, programmer 50may be used to transmit initial programming information to IMD 18. Forexample, the clinician may store therapy programs within IMD 18 with theaid of programmer 50. Programmer 50 may assist the clinician in thecreation/identification of therapy programs by providing a methodicalsystem for identifying potentially beneficial therapy parameter values.Programmer 50 may also, or alternatively, be configured for use bypatient 12. When configured as a patient programmer, programmer 50 mayhave limited functionality (compared to a clinician programmer) in orderto prevent patient 12 from altering critical functions of IMD 18 orapplications that may be detrimental to patient 12. In this manner,programmer 50 may only allow patient 12 to adjust values for certaintherapy parameters or set an available range of values for a particulartherapy parameter.

Programmer 50 may provide information to the clinician and/or patient12. For example, programmer 50 may present an indication to patient 12when therapy is being delivered, when patient 12 input has triggered achange in therapy or when the power source within programmer 50 or IMD18 needs to be replaced or recharged. For example, programmer 50 mayinclude an alert LED, may flash a message to patient 12 via a programmerdisplay, or generate an audible sound or somatosensory cue to confirmpatient input was received, e.g., to indicate a patient state or tomanually modify a therapy parameter. In addition, programmer 50 maypresent information to patient 12 and/or the clinician duringimplantation of IMD 18 and/or programming of IMD 18.

In addition to monitoring and/or providing therapy to patient 12 via IMD18, system 40 may include additional devices configured to detectgesture input and control one or more actions of a medical procedurebased on the detected gesture input. System 40 may thus include agesture interface that incorporates data obtained by IMD 18. Similar tosystem 10 of FIG. 1, sensors 30 (e.g., camera-based sensors) may bepositioned in such a manner to capture the movements of one or moreanatomical regions of patient 12, such as one or both of hands 52.Sensors 30 may also be configured to detect a wearable device fordetection of gestures, such as gesture device 54. Sensors 30 may bepositioned to detect hands 52A, 52B within an envelope of space that mayor may not include a portion of patient 12. However, the envelope may beselected such that hands 52 must be placed within the envelope adjacentto patient 12 to reduce the possibility that unintended users movementsare detected as unintended gesture inputs.

Gesture control device 22 may be in communication with sensors 30 and/orcontrol the operation of sensors 30. For example, gesture control device22 may receive field input data (e.g., raw video data) or gesture inputs(e.g., processed raw video data) from sensors 30. As discussed in FIG.1, gesture control device 22 may determine various gesture inputs basedon the information received from sensors 30 and control, based on thegesture input received, one or more actions of a medical procedure. Forexample, gesture control device 22 may control an implantation devicethat assists a clinician to insert leads 46 into patient 12, an imagingdevice that assists the clinician in determining the location of theimplanted lead within patient 12, or even external programmer 50 and/orIMD 18 in programming initial parameter values during the implantationprocess for leads 46 and IMD 18.

However, in the example of FIG. 2, gesture control device may receiveadditional information regarding gesture input to ensure the anticipateduser has provided the gesture and/or confirm that the gesture inputdetected by sensors 30 was correct and intended by the user. Forexample, IMD 18 may generate gesture information and transmit theinformation to gesture control device 22 via external programmer 50.Since external programmer 50 may be in communication with gesturecontrol device 22 and IMD 18, IMD 18 may transmit gesture information toexternal programmer 50, and external programmer 50 may then transmit thegesture information to gesture control device 22. The gestureinformation may be unprocessed by external programmer 50, or externalprogrammer 50 may process the signals from IMD 18 and generate thegesture information sent to gesture control device 22.

IMD 18 may be configured to detect gestures or the presence of hands 52Aand 52B (collectively “hands 52”) and/or gesture device 54. For example,leads 46 may transmit changes in electrical fields caused by thepresence of hands 52 of a user (e.g., user 14), the presence of wearablegesture device 54, or changes in tissue from patient 12 to IMD 18. Thesechanges may be indicative of the presence of one or both of hands 52 todetect gesture input and/or confirm that the gesture input was detectedby an expected user in the vicinity of patient 12. One or more of leads46 may also include one or more sensors configured to detect thepresence of hands 52 and/or gesture device 54. In this manner, leads 46may carry electrodes and/or other types of sensors configured to monitorpatient 12, provide therapy to patient 12, and/or detect gestures fromthe user and/or patient 12.

Using the signals received from leads 46 and/or any other sensors, IMD18 may generate gesture information representative of the presenceand/or motions of hands 52 and transmit the gesture information toexternal programmer 50 en route to gesture control device 22. IMD 18 maydetect gestures and/or generate gesture information periodically, inresponse to specific steps in a medical procedure, or in response to arequest received from gesture control device 22. For example, gesturecontrol device 22 may determine that a gesture control mode has beenentered and transmit a request to IMD 18 to begin transmitting gestureinformation back to gesture control device 22 for the duration of thegesture control mode. Although IMD 18 may transmit gesture informationto gesture control device 22 via external programmer 50, IMD 18 maydirectly transmit gesture information to gesture control device 22 inother examples. In alternative examples, external programmer 50 mayperform the functions of gesture control device 22 and control medicaldevices based on the gesture input received from sensors 30 and/or IMD18.

Gesture control device 22 may control actions of a medical procedureaccording to field input data received from sensors 30 and gestureinformation received from IMD 18. For example, gesture control device 22may determine gesture input based on both the information from sensors30 and IMD 18. In some examples, gesture control device 22 may receivethe gesture input from sensors 30 (e.g., field input data) and confirmthat the gesture input was from gestures made by one or more hands 52close to patient 12. In other words, IMD 18 may monitor a change inelectrical field from leads 46 or receive data indicative of thepresence of hands 52 that indicates the gestures were made by theclinician working with patient 12. Gesture control device 22 may thususe the gesture information from IMD 18 as a confirmation that thegesture input was not inadvertent or from an unintended user. In someexamples, gesture control device 22 may compare the field input datafrom sensors 30 to the gesture information from IMD 18 to determine ifthe field input data correlates with the gesture information. Althoughthe field input data may be more detailed (e.g., fine resolution)regarding the conformation and/or movements of hands 52, the gestureinformation from IMD 18 may indicate coarse resolution regarding themovement or presence of hands 52. Therefore, the gestures detected fromthe field input data from sensors 30 may correlate to the general coarserepresentation of the gestures detected by IMD 18. Gesture controldevice 22 may thus use the gesture information from IMD 18 to confirm orotherwise increase the probability that the gesture input is accurateand from the intended user. If gesture control device 22 does notdetermine that the field input data correlates with gesture informationfrom IMD 18, gesture control device 22 may discard any gesture inputfrom the field input data or otherwise withhold any action based on thefield input data because the gesture information did not confirm thegesture input.

In this manner, field input data from sensors external from patient 12may be particularly useful to correlate gestures provided in spatialrelation to a specific location of patient anatomy or medical componentsimplanted within patient 12. This correlation may be used by gesturecontrol device 22 to change an aspect of therapy at that particularlocation of the patient. For example, IMD 18 may detect the location ofone or more hands 52 with respect to an anatomical landmark and/orcomponent associated with IMD 18 (e.g., one or more electrodes of a leadconnected with IMD 18), and field input data from sensors 30 may be usedto detect gesture input at that location. The gesture input may requesta change to a stimulation parameter (e.g., increase or decreasestimulation intensity, pulse width and/or pulse rate and/or change theselected electrode configuration used to deliver stimulation therapy) orindicate a certain beneficial effect or side effect of therapy at thatlocation of patient anatomy. IMD 18 and/or gesture control device 22 maythus associate the change in therapy or other information from thegesture input to the detected anatomical and/or component position andgenerate an instruction related to therapy. For example, the instructionmay be a change to a therapy parameter at that identified location ofanatomy or identified medical component or a notation regarding thetherapy that is associated to the anatomical location. The instructionor notation may be stored by IMD 18 and/or another device of system 10.In this manner, system 10 may receive instructions based on both thefield input data from sensors 30 and gesture information from IMD 18.

In some examples, IMD 18 may generate the gesture information from thelocation of gesture device 54. The clinician may wear gesture device 54to identify one or both of hands 52 as the hands of the clinician. IMD18 may include a transceiver or other communication device that detectswhen gesture device 54 is within a predetermined distance of IMD 18.Gesture device 54 may include a similar transponder or circuitry toallow IMD 18 and gesture device 54 to determine when the gesture deviceis within the predetermined distance of IMD 18. An example transponderor communication protocol may include Bluetooth protocol, near-fieldcommunication protocol, or any other one-way or two-way communicationprotocol. In other examples, gesture device 54 may detect that it iswithin a predetermined distance from IMD 18 and/or leads 46 and transmitthat information to external programmer 50 and/or gesture control device22 for use by gesture control device in confirming the detected gestureinput. In some examples, gesture device 54 may include a motion sensingdevice (e.g., accelerometer and/or gyroscope) that provide additionalinformation regarding the movement of hand 52.

Gesture device 54 may be constructed in the form of a band, bracelet,ring, attachable patch, or even as part of a glove worn by one or bothof hands 52. Gesture device 54 may be a passive device that is onlydetectable by another device (e.g., a device that reflects light, altersan electric field, or a passive near-field communication circuit). Inthis manner, gesture device 54 may be a device configured to improve theefficiency or accuracy of external components (e.g., sensors 30) orimplanted components (e.g., IMD 18) for detection of gesture inputs(e.g., reflective elements for modalities such as optical detection orresonant circuits for acoustic systems). Alternatively, gesture device54 may include circuitry that provides active communication with anotherdevice, such as IMD 18, external programmer 50, and/or gesture controldevice 22. Although gesture device 54 is shown on only hand 52B, anothergesture device 54 may be worn on hand 52A, or a plurality of gesturedevices may be worn on respective fingers of one or both of hands 52.

In configurations where gesture information from IMD 18 only provides acoarse representation of the presence of hands 52, the gestureinformation may only be used as a confirmation of gesture input (e.g.,used in conjunction with more fine resolution data such as field inputdata from sensors 30). However, coarse gesture information and/or morefine resolution gesture information from IMD 18 may be used alone bygesture control device 22 to determine gesture input for control ofmedical procedure actions. In other words, field input data from sensors30 may not be necessary in some circumstances. For example, IMD 18 maydetect positions and/or movements of one or both of hands 52 withrespect to IMD 18 and/or electrodes or other sensors carried by leads46. IMD 18 may generate this gesture information and transmit thegesture information to external programmer 50 and/or gesture controldevice 22 to allow gesture control device 22 to control an actionrelated to the medical procedure. Alternatively, IMD 18 may directly usethis gesture information to adjust one or more parameter values thatdefines therapy delivered by IMD 18 and/or send the gesture informationto external programmer 50. For example, IMD 18 may be configured todetect relative hand movement up or down an implanted lead to providegesture information that correlates to an increase or decrease,respectively, of stimulation intensity. However, the detection ofabsolute hand position with respect to one or more leads or otherimplanted components may also be used to provide gesture information. Inany example, IMD 18 may generate gesture information that is itselfsufficient to detect at least some types of gesture input from a user.

Gesture information from IMD 18 may be particularly useful for gesturecontrol of IMD 18 since only the clinician positioned near patient 12will be control over IMD 18. In addition, when IMD 18 is beingimplanted, the clinician may be near patient 12 and it may be desirablenot to physically touch any other devices during the procedure. Forexample, skin opening 48 may be susceptible to infection due to lead 46and IMD 18 implantation. During this time, gesture input control of oneor more aspects of the medical procedure may reduce the risk ofinfection and/or contamination of handheld devices. Moreover, IMD 18detection of hands 52 may cause gesture control device 22 to switchcontrol of devices using gesture input. For example, gesture controldevice 22 may control a surgical device using gestures from hands 52. Inresponse to receiving gesture information from IMD 18 that indicates thepresence of hands 52 near IMD 18, gesture control device 22 may switchgesture input control from the surgical device to control of IMD 18(e.g., adjusting one or more parameters that defines the operation ofIMD 18).

In some examples, gesture information from IMD 18 may be used as anindicator gesture input and/or termination gesture input. For example,in response to receiving gesture information from IMD 18 that indicateshands 52 have been detected, gesture control device 22 may enter thegesture control mode for control of IMD 18 using field input data fromsensors 30. Similarly, hands 52 no longer detected by the gestureinformation may be a termination gesture input that causes gesturecontrol device 22 to exit the gesture control mode.

System 40 is generally described for use during implantation of IMD 18.However, IMD 18 may generate gesture information in other situations aswell. For example, IMD 18 may be used by a gesture control device duringa programming session for IMD 18 in a clinician's office. IMD 18 maygenerate gesture information during the programming session and transmitthe gesture information to external programmer 50 or gesture controldevice 22 for facilitating gesture input control during the programmingsession.

Although IMD 18 is generally described in FIG. 2, techniques of thisdisclosure may also be applicable to external or partially externalmedical device in other examples. For example, IMD 18 may instead beconfigured as an external medical device coupled to one or morepercutaneous medical leads or even externally attached leads or devices.The external medical device may be a chronic, temporary, or trialelectrical stimulator. In addition, an external electrical stimulatormay be used in addition to one or more IMDs 18 to deliver electricalstimulation described herein. External medical devices may similarly beconfigured to detect the presence of hands 52 and/or gesture device 54to detect gesture input, assist in the detection of gesture input,and/or confirm the presence of hands 52 or gesture device 54 that aidsthe determination of gesture input by gesture control device 22.

FIG. 3 is a block diagram of example gesture control device 22 of FIGS.1 and 2. FIG. 3 illustrates only one particular example of gesturecontrol device 22, and many other example embodiments of gesture controldevice 22 may be used in other instances. For example, gesture controldevice 22 may include additional components and run multiple differentapplications. Gesture control device 22 may be configured to obtainand/or generate gesture input from sensors 30A and 30B. Gesture controldevice 22 may also generate commands for transmission to other devicesfor control during a medical procedure.

As shown in FIG. 3, gesture control device 22 may include processor 60,memory 62, user interface 66, communication module 68, and power source69. Sensors 30 may be located separately from gesture control device 22and in communication with gesture control device 22 and processor 60. Inother examples, sensors 30 may be incorporated within gesture controldevice 22. Each of components 30A, 30B, 60, 62, 66, 68 and 69 may beinterconnected (physically, communicatively, and/or operatively) forinter-component communications and functionality.

Processor 60, in one example, is configured to implement functionalityand/or process instructions for execution, such as controlling sensors30, receiving field input data 64, and determining gesture input. Fieldinput data 64 may include the data received from sensors 30 and may bestored in memory 62 for temporary and/or long term storage. Processor 60may also be configured to process instructions stored within memory 62.Processor 60 may also be configured to generate metadata or supplementaldata (e.g., time stamps or any other related information) to field inputdata 64 and/or gesture inputs and store such data with the video framesas the field input data 64.

Memory 62, in one example, is configured to store information withingesture control device 22 during operation. Memory 62, in some examples,is described as a computer-readable storage medium. Memory 62 may alsobe described as a storage device or computer-readable storage device. Insome examples, memory 62 is a temporary memory, meaning that a primarypurpose of memory 62 is not long-term storage. However, memory 62 mayalso be described as non-transitory. Memory 62, in some examples, may bedescribed as a volatile memory, meaning that memory 62 does not maintainstored contents when the computer is turned off. Examples of volatilememories include random access memories (RAM), dynamic random accessmemories (DRAM), static random access memories (SRAM), and other formsof volatile memories known in the art. In some examples, memory 62 isused to store program instructions for execution by processor 60.

Gesture control device 22, in some examples, also includes acommunication module 68. Gesture control device 22, in one example,utilizes communication module 68 to communicate with other computingdevices (e.g., networked server 106 of FIG. 7), programmers (e.g.,programmer 50 of FIG. 2), sensors 30 in some examples, medical devices(e.g., IMD 18 and/or medical device 112), or more networks, such asnetwork 110 shown in FIG. 7. In this manner, gesture control device 22may transmit field input data 64 to other computing devices foranalysis, receive gesture control information resulting from the fieldinput data, and/or transmit control information to other medical devicesto be controlled. Communication module 68 may be a network interfacecard, such as an Ethernet card or other wired interface. In otherexamples, communication module 68 may include an optical transceiver, aradio frequency transceiver, or any other type of device that can sendand receive information. Other examples of such communication modulesmay include Bluetooth, 3G and WiFi radios in mobile computing devices aswell as USB. In some examples, gesture control device 22 utilizescommunication module 68 to wirelessly communicate with another computingdevice (e.g., external programmer 50 of IMD 18 of FIG. 2) or one or morenetworks.

Gesture control device 22, in one example, also includes one or moreuser interface 66. User interface 66 may include a touch-sensitivescreen, mouse, a keyboard, a voice responsive system, camera, or anyother type of device for detecting a command from a user. In oneexample, user interface 66 may include a touch-sensitive screen, soundcard, a video graphics adapter card, or any other type of device forconverting a signal into an appropriate form understandable to humans ormachines. In addition, user interface 66 may include a speaker, acathode ray tube (CRT) monitor, a liquid crystal display (LCD), or anyother type of device that can generate intelligible output to a user.For example, user interface 66 includes output device 24 (e.g., adisplay device such as an LCD) and output device 26 (e.g., an audiodevice such as one or more speakers). User interface 66 may also includeone or more input devices (e.g., a touch-sensitive screen, a pointingdevice, and/or a keyboard).

Gesture control device 22, in some examples, includes one or more powersources 69, which provide power to gesture control device 22. Generally,power source 69 may utilize power obtained from a wall receptacle orother alternating current source. However, in other examples, powersource 69, may include one or more rechargeable or non-rechargeablebatteries (e.g., constructed from nickel-cadmium, lithium-ion, or othersuitable material). In other examples, power source 69 may be a powersource capable of providing stored power or voltage from another powersource.

FIG. 4 is a block diagram of example IMD 18 of FIGS. 1 and 2. As shownin the example of FIG. 4, IMD 18 includes processor 70, therapy module72, power source 75, memory 71, telemetry module 73, and sensor module74. In other examples, IMD 18 may include a greater or fewer number ofcomponents. In general, IMD 18 may comprise any suitable arrangement ofhardware, alone or in combination with software and/or firmware, toperform the various techniques described herein attributed to IMD 18 andprocessor 70. In various examples, IMD 18 may include one or moreprocessors 30, such as one or more microprocessors, digital signalprocessors (DSPs), application specific integrated circuits (ASICs),field programmable gate arrays (FPGAs), or any other equivalentintegrated or discrete logic circuitry, as well as any combinations ofsuch components. IMD 18 also, in various examples, may include a memory71, such as random access memory (RAM), read only memory (ROM),programmable read only memory (PROM), erasable programmable read onlymemory (EPROM), electronically erasable programmable read only memory(EEPROM), flash memory, comprising executable instructions for causingthe one or more processors to perform the actions attributed to them.Moreover, although processor 70, therapy module 72, and telemetry module73 are described as separate modules, in some examples, processor 70,therapy module 72, and telemetry module 73 may be functionallyintegrated. In some examples, processor 70, therapy module 72, andtelemetry module 73 may correspond to individual hardware units, such asASICs, DSPs, FPGAs, or other hardware units.

Memory 71 (e.g., a storage device) may store therapy programs or otherinstructions that specify therapy parameter values for the therapyprovided by therapy module 72 and IMD 18. In some examples, memory 71may also store instructions for communication between IMD 18 andprogrammer 50, or any other instructions required to perform tasksattributed to IMD 18. Memory 71 may also store instructions fordetecting gestures (e.g., the presence of a hand or gesture device asdescribed in FIG. 2) and generating gesture information to aid in theerror mitigation techniques described herein.

Generally, therapy module 72 may generate and deliver electricalstimulation under the control of processor 70. In some examples,processor 70 controls therapy module 72 by accessing memory 71 toselectively access and load at least one of the stimulation programs totherapy module 72. For example, in operation, processor 70 may accessmemory 71 to load one of the stimulation programs to therapy module 72.In such examples, relevant stimulation parameters may include a voltageamplitude, a current amplitude, a pulse frequency, a pulse width, a dutycycle, one or more spatial electrode movement patterns that define thecombination of electrodes 47A carried by lead 46A or electrodes 47Bcarried by lead 46B that therapy module 72 uses to deliver theelectrical stimulation signal. Although therapy module 72 may beconfigured to generate and deliver electrical stimulation therapy viaone or more of electrodes 47A or 47B, a different therapy module may beconfigured to provide different therapy to patient 12, such as drugdelivery therapy via a catheter. These and other therapies may beprovided by IMD 18. Although four electrodes are shown as included inelectrodes 47A and four electrodes are shown as included in electrodes47B, fewer or greater number of electrodes may be provided in otherexamples.

An exemplary range of electrical stimulation parameters that may be usedto deliver effective treatment for chronic pain, e.g., when applied tospinal cord 42, are listed below. While stimulation pulses aredescribed, stimulation signals may be of any of a variety of forms suchas sine waves or the like.

Pulse Frequency: between approximately 0.5 Hz and 10,000 Hz. In oneexample, pulse frequency may be between approximately 5 Hz and 250 Hz orbetween approximately 30 Hz and 130 Hz. In other examples, pulsefrequency may be greater than 250 Hz or even greater than 1,000 Hz.Pulse frequencies greater than 1,000 Hz may be considered to be greaterthan the nerve firing potential of affected nerve fibers to inhibitnerve firing. However, in some examples, the pulse frequency may bebetween approximately 1,000 Hz and 10,000 Hz.

Amplitude: between approximately 0.1 volts and 50 volts, more preferablybetween approximately 0.5 volts and 20 volts, and still more preferablybetween approximately 1 volt and 10 volts. In other examples, a currentamplitude may be defined as the biological load in the voltage that isdelivered. For example, the range of current amplitude may be betweenapproximately 0.1 milliamps (mA) and 50 mA. In other examples, currentamplitude may be between approximately 1.0 mA and 10 mA.

Pulse Width: between about 10 microseconds and 5000 microseconds. Insome examples, the pulse width may be between approximately 100microseconds and 1000 microseconds or between approximately 180microseconds and 450 microseconds. With higher frequency pulses, thepulse width may be smaller to accommodate the increased frequency. Forexample, the pulse width may be between approximately 10 microsecondsand 50 microseconds.

Power source 75 may include one or more capacitors, batteries, or otherenergy storage devices. IMD 18 may thus also include an inductive coiland a recharge module (both not shown) configured to manage therecharging session for power source 75. Although inductive coupling maybe used to recharge power source 75, other wireless energy transfertechniques may alternatively be used. Alternatively, power source 75 maynot be rechargeable.

Processor 70 may also control the exchange of information withprogrammer 50, gesture device 54, and/or gesture control device 22 usingtelemetry module 73. Telemetry module 73 may be configured for wirelesscommunication using radio frequency protocols or inductive communicationprotocols. Telemetry module 73 may include one or more antennasconfigured to communicate with programmer 50, for example. Processor 70may transmit operational information and receive therapy programs ortherapy parameter adjustments via telemetry module 73. Also, in someexamples, IMD 18 may communicate with other implanted devices, such asstimulators, control devices, or sensors, via telemetry module 73. Forexample, telemetry module 73 may receive user input, spatial electrodemovement patterns, or other commands from programmer 50. Accordingly,telemetry module 73 may send information to external programmer 50 on acontinuous basis, at periodic intervals, or upon request from IMD 18 orprogrammer 50.

Sensor module 74 may be configured to detect position and/or motion ofIMD 18, the presence of any hands of a user, or the presence of gesturedevice 54. For example, sensor module 74 may include one or moreaccelerometers, one or more gyroscopes, or any other movement detectiondevices. Sensor module 74 may additionally, or alternatively, includeone or more ultrasound transducers, capacitive sensors, acousticsensors, strain sensors, pressure sensors (e.g., on the surface of ahousing of IMD 18), optical sensors, thermal sensors, or communicationdevices that may be configured to detect movement of hands or otherwiseprovide data related to gestures performed by a user. Any of all ofthese sensors may be used by sensor module 74 to detect gestures. IMD 18may be configured to detect gesture input itself via the techniquesdiscussed herein, without the use of information provided by otherexternal sensors such as sensors 30 of system 10.

Although IMD 18 is described as providing spinal cord stimulation, IMD18 may be configured to provide any number of other electricalstimulation therapies. For example, IMD 18 and leads 46 may beconfigured to provide deep brain stimulation, peripheral nervestimulation, gastric stimulation, pelvic floor stimulation, or any othertype of stimulation.

FIG. 5 is a block diagram of example external programmer 50 of FIG. 2.Although programmer 50 may generally be described as a hand-held device,programmer 50 may be a larger portable device or a more stationarydevice. In addition, in other examples, programmer 50 may be included aspart of an external charging device or include the functionality of anexternal charging device. As illustrated in FIG. 5, programmer 50 mayinclude a processor 76, memory 77, user interface 78, telemetry module79, and power source 80. Memory 77 may store instructions that, whenexecuted by processor 76, cause processor 76 and external programmer 50to provide the functionality ascribed to external programmer 50throughout this disclosure. For example, processor 76 may be configuredto transmit selected therapy parameter sets that at least partiallydefine the operation of IMD 18 to IMD 18.

In general, programmer 50 comprises any suitable arrangement ofhardware, alone or in combination with software and/or firmware, toperform the techniques attributed to programmer 50, and processor 76,user interface 78, and telemetry module 79 of programmer 50. In variousexamples, programmer 50 may include one or more processors, such as oneor more microprocessors, DSPs, ASICs, FPGAs, or any other equivalentintegrated or discrete logic circuitry, as well as any combinations ofsuch components. Programmer 50 also, in various examples, may include amemory 77, such as RAM, ROM, PROM, EPROM, EEPROM, and flash memory (anda storage device such as a hard disk or a CD-ROM in some examples),comprising executable instructions for causing the one or moreprocessors to perform the actions attributed to them. Moreover, althoughprocessor 76 and telemetry module 79 are described as separate modules,in some examples, processor 76 and telemetry module 79 are functionallyintegrated. In some examples, processor 76 and telemetry module 79correspond to individual hardware units, such as ASICs, DSPs, FPGAs, orother hardware units.

Memory 77 (e.g., a storage device) may store instructions that, whenexecuted by processor 76, cause processor 76 and programmer 50 toprovide the functionality ascribed to programmer 50 throughout thisdisclosure. For example memory 77 may include instructions that causeprocessor 76 to obtain a parameter set from memory, select a spatialelectrode movement pattern, or receive a user input and send acorresponding command to IMD 14, or instructions for any otherfunctionality. In addition, memory 77 may include a plurality ofprograms, where each program includes a parameter set that definesstimulation therapy

User interface 78 may include a button or keypad, lights, a speaker forvoice commands, a display, such as a liquid crystal (LCD),light-emitting diode (LED), or organic light-emitting diode (OLED). Insome examples the display may be a touch screen. User interface 78 maybe configured to display any information related to the delivery ofstimulation therapy, identified patient behaviors, sensed patientparameter values, patient behavior criteria, or any other suchinformation. User interface 78 may also receive user input via userinterface 78. The input may be, for example, in the form of pressing abutton on a keypad or selecting an icon from a touch screen. The inputmay request starting or stopping electrical stimulation, the input mayrequest a new spatial electrode movement pattern or a change to anexisting spatial electrode movement pattern, or the input may requestsome other change to the delivery of electrical stimulation.

Telemetry module 79 may support wireless communication between IMD 14and programmer 50 under the control of processor 76. Telemetry module 79may also be configured to communicate with another computing device viawireless communication techniques, or direct communication through awired connection. In some examples, telemetry module 79 may besubstantially similar to telemetry module 73 of IMD 18 described herein,providing wireless communication via an RF or proximal inductive medium.In some examples, telemetry module 79 may include an antenna, which maytake on a variety of forms, such as an internal or external antenna.

Examples of local wireless communication techniques that may be employedto facilitate communication between programmer 50 and other devicesinclude RF communication according to the 802.11, Bluetoothspecification sets, or other standard or proprietary telemetryprotocols. In this manner, other external devices may be capable ofcommunicating with programmer 50 without needing to establish a securewireless connection. As described herein, telemetry module 79 may beconfigured to transmit a spatial electrode movement pattern or otherstimulation parameter values to IMD 18 for delivery of stimulationtherapy. Telemetry module 79 may utilize these wireless communicationtechniques to communicate with IMD 18 and/or other wirelesscommunication protocols that provide a secure connection with IMD 18.

In some examples, external programmer 50 may be configured to performthe features attributed to gesture control device 22 described herein.For example, external programmer 50 may receive field input data fromsensors 30 of FIG. 2 and determine gesture input from the field inputdata. In response to determining gesture input, processor 76 maydetermine corresponding actions such as adjustments to parameter valuesand transmit the adjustments to IMD 18.

FIG. 6 is a block diagram of an example networked server 106 andrepository 46 of FIG. 7. FIG. 6 illustrates only one particular exampleof server 106, and many other example embodiments of server 106 may beused in other instances. For example, server 106 may include additionalcomponents and run multiple different applications. Although gesturecontrol device 22 is generally described as determining gesture inputand controlling one or more actions (e.g., one or more medical devices)for a medical procedure, server 106 may analyze field input data todetermine gesture input and/or identify commands based on the receivedgesture input. Gesture control device 22 may be configured to transmitfield input data, for example, to server 106 for analysis and processingto determine the gesture inputs received from a user. For example,server 106 may be configured to perform some or all of the processesdescribed with respect to FIGS. 11-18.

As shown in the specific example of FIG. 6, server 106 may includeand/or house one or more processors 82, memory 83, a network interface84, user interface 86, gesture detection module 88, and power source 90.Server 106 may be in communication with repository 108, such thatrepository 108 is located external of server 106. In other examples,repository 108 may include one or more storage devices within anenclosure of server 106. Server 106 may also include an operatingsystem, which may include modules and/or applications that areexecutable by processors 82 and server 106. Each of components 82, 83,84, 86, 88, and 90 may be interconnected (physically, communicatively,and/or operatively) for inter-component communications.

Processors 82, in one example, are configured to implement functionalityand/or process instructions for execution within server 106, such asdetermining gesture inputs from field input data and/or determining thecontrols corresponding to the received gesture input. Processors 82 mayalso, or alternatively, process gesture information from IMD 18 todetermine whether or not a user's hand has provided a gesture or isotherwise present within an envelope near IMD 18 or leads 46. Forexample, processors 82 may be capable of processing instructions storedin memory 83 or instructions stored in repository 108. Theseinstructions may define or otherwise control the operation of server106. In some examples, gesture detection module 88 (which may includeone or more dedicated processors) may be configured to analyze the fieldinput data 92 (which may or may not be the same field input data 64described in FIG. 3) and identify the gesture input received from theuser within field input data 92.

Memory 83, in one example, is configured to store information withinserver 106 during operation. Memory 83, in some examples, is describedas a computer-readable storage medium. Memory 83 may also be describedas a storage device or computer-readable storage device. In someexamples, memory 83 is a temporary memory, meaning that a primarypurpose of memory 83 is not long-term storage. However, memory 83 mayalso be described as non-transitory. Memory 83, in some examples, may bedescribed as a volatile memory, meaning that memory 83 does not maintainstored contents when the computer is turned off. Examples of volatilememories include random access memories (RAM), dynamic random accessmemories (DRAM), static random access memories (SRAM), and other formsof volatile memories known in the art. In some examples, memory 83 isused to store program instructions for execution by processors 82.Memory 83, in one example, is used by software or applications runningon server 106 to temporarily store information during program execution.Although memory 83 of FIG. 6 is not described as including field inputdata 92 or gesture actions 96, for example, memory 83 may store suchinstructions and other data in other examples.

Repository 108, in some examples, also includes one or morecomputer-readable storage media, such as one or more storage devices.Repository 108 may be configured to store larger amounts of informationthan memory 83. Repository 108 may further be configured for long-termstorage of information. In some examples, repository 108 may includenon-volatile storage elements. Examples of such non-volatile storageelements include magnetic hard discs, optical discs, floppy discs, flashmemories, or forms of electrically programmable memories (EPROM) orelectrically erasable and programmable (EEPROM) memories.

Repository 108 may be configured to store information related to orcollected from each of multiple patients. For example, repository 108may be configured to store field input data 92 collected from one ormore patients. The field input data 92 may include the raw data, orprocessed data, obtained from sensors 30 or other sensors configured todetect gesture input. Repository 108 may also include gesture inputtypes 94, gesture actions 96, user specific information 98, gesturecontrol schemes 100, and medical procedures 102. The data stored inrepository 108 may also be stored in memory 62 of gesture control device22 of FIG. 3, for example.

Gesture input types 94 may define the different types of gesture inputthat may be identified or determined by server 106. For example, gestureinput types may include one or more indicator gesture input types, oneor more control gesture input types, one or more confirmation gestureinput types, and one or more termination gesture input types. A gestureinput type may define a gesture that is used for a specific purpose. Forexample, a gesture type may be a hand with five outstretched fingers, aclenched fist, or the pinching motion of a pointer finger moving towardsthe thumb. Other example gesture types may include directional swipes(e.g., up, down, left, or right), hand or finger rotations (e.g.,clockwise for an increase in a parameter such as amplitude orcounter-clockwise for a decrease in a parameter such as amplitude), andchanging between close and far away from the sensor to control anadjustment to a parameter. Relative movement gestures may be more easilydetected by an IMD than more stationary gestures distinguishable byexternal sensors such as optical sensors. Each of these gesture inputtypes 94 may be used by a user to request a specific input related tothe gesture input type. In other words, gesture input types 94 mayinclude a library of the different gestures detectable by server 106.

Gesture actions 96 may include a database of actions, or controls, thatcorrespond to respective gesture input types. For example, gestureactions 96 may include such actions as insertion of a lead into patient12 by a surgical device, turning ablation energy on and off, orincreasing and decreasing the parameter value of electrical stimulationvoltage. Gesture actions 96 may include each of the actions, orcontrols, and a link to the corresponding gesture input type that can beused to define or request each of the actions.

User specific gesture information 98 may include any information used toidentify a particular user (e.g., a specific clinician) from the fieldinput data 92 or determined gesture input. Thus, user specific gestureinformation 98 may include calibrated parameters or factors from whichserver 106 can determine whether or not a particular gesture input wasprovided by the specific user. In one example, a user may providegestures having certain sizes, speeds, angles, or positions within anenvelope. User specific gesture information 98 may quantify thesespecific features of each gesture provided by the user such that thegesture input is similar to a “fingerprint” of the user for the gestureinput. Instead of merely determining that an outstretched hand andfingers or movement of a hand from one lower position to a higherposition are a certain gesture input type, server 106 may be configuredto determine what user provided those gesture inputs based on the userspecific gesture information stored in repository 108. For example,different users may have respective different permissions or levels ofallowable control using gesture inputs. In this manner, server 106 (oranother device) may determine that a correct gesture was performed, butidentify the gesture as provided by the wrong user (e.g., the wrong userbeing a user not identified as having permission to provide the detectedgesture or a user different from the user that entered the gesturecontrol mode). A user may need to perform a calibration session, or haveprevious gesture input analyzed, so that server 106 or another computingdevice can generate the user specific gesture information 98 for thatparticular user.

Gesture control schemes 100 may be a list of possible control schemesusing gestures and algorithms needed to perform each scheme. A gesturecontrol scheme may define how server 106 will control a medical deviceor another action for a medical procedure based on received gestures.For example, a gesture control scheme may require that an indicatorgesture input must be received and a gesture control mode entered beforeserver 106 can receive control gesture input and perform an action.Another gesture control scheme may include requiring a control gestureinput and a confirmation gesture input to be received simultaneously inorder for server 106 to control a device to perform an action. Each stepof a medical procedure or process that can receive gesture input maylink to one or more gesture control schemes 100 to identify how gestureinput can be used for control of one or more devices.

Medical procedures 102 may include those medical procedures in whichgesture input can be used to control one or more actions. Each of themedical procedures 102 may define one or more steps of the procedure andidentify which step can accept gesture input to perform an action. Inaddition, each of medical procedures 102 may identify which gesturecontrol schemes 100 can be used for control, such as whether or not agesture control mode can be entered upon the start of a step or whetherthe gesture control mode should be exited in response to a step beingcompleted. Gesture detection module 88 may incorporate one or more ofthe rules stored in repository 108 in determining the gesture input andhow to control any actions based on the received gesture input. Therules and schemes described herein may be implemented to minimize anyerror related to gesture input control of actions related to medicalprocedures.

Server 106, in some examples, also includes a network interface 84.Server 106, in one example, utilizes network interface 84 to communicatewith other computing devices (e.g., gesture control device 22 of FIG.1), programmers (e.g., programmer 50 of FIG. 2), medical devices, ormore networks, such as network 110 shown in FIG. 7. In this manner,server 106 may receive field input data 92 and transmit information suchas gesture control information 114 (shown in FIG. 7). Network interface84 may be a network interface card, such as an Ethernet card or otherwired interface. In other examples, network interface 84 may include anoptical transceiver, a radio frequency transceiver, or any other type ofdevice that can send and receive information. Other examples of suchnetwork interfaces may include Bluetooth, 3G, 4G and WiFi radios inmobile computing devices as well as USB. In some examples, server 106utilizes network interface 84 to wirelessly communicate with anothercomputing device (e.g., gesture control device 22 of FIG. 1) or othernetworked computing devices.

Server 106, in one example, also includes one or more user interfaces86. User interface 86 may include a touch-sensitive screen, mouse, akeyboard, a voice responsive system, camera, or any other type of devicefor detecting a command from a user. In one example, user interface 86may include a touch-sensitive screen, sound card, a video graphicsadapter card, or any other type of device for converting a signal intoan appropriate form understandable to humans or machines. In addition,user interface 86 may include a speaker, a cathode ray tube (CRT)monitor, a liquid crystal display (LCD), or any other type of devicethat can generate intelligible output to a user.

Server 106, in some examples, includes one or more power sources 90,which provide power to server 106. Generally, power source 90 mayutilize power obtained from a wall receptacle or other alternatingcurrent source. However, in other examples, power source 90, may includeone or more rechargeable or non-rechargeable batteries (e.g.,constructed from nickel-cadmium, lithium-ion, or other suitablematerial). In other examples, power source 90 may be a power sourcecapable of providing stored power or voltage from another power source.

In other examples, one or more of processors 82 may perform some or allof the functions described herein within respect to gesture detectionmodule 88. Any software implemented within or executed by server 106 maybe implemented or contained within, operable by, executed by, and/or beoperatively/communicatively coupled to components of server 106 (e.g.,processors 82, memory 83, network interface 84, and/or repository 108).

FIG. 7 is a conceptual diagram illustrating example system 104 thatincludes networked server 106 of FIG. 6 for determining gesture inputbased on received field input data 64. As shown in FIG. 7, system 104includes gesture control device 22, sensors 30, network 110, networkedserver 106 (e.g., a computing device), repository 108, and externalprogrammer 50, and medical device 112. Gesture control device 22, insome examples, is or is a part of a computing device (e.g., aworkstation, a netbook computer, a notebook computer, a tablet computingdevice, or server computing device). In any examples, gesture controldevice 22 may be a computing device configured to assist in theobtaining of gesture input and/or control of an action based on thegesture input. Gesture control device 22 may also include a displaydevice (e.g., output device 24 of FIG. 3) and be configured to controlthe display device. The display device be housed by gesture controldevice 22 or be external to gesture control device 22. Although sensors30 may be separate devices in communication with gesture control device22, sensors 30 may be coupled to or at least partially within a housingof gesture control device 22 in other examples. In some examples,gesture control device 22 may receive field input data 64 from sensors30, detect gesture input from the field input data 64, determine actionsfor a medical device (e.g., medical device 112) corresponding to thegesture input, and transmit control signals to the medical device basedon the determined actions. However, as described in FIG. 7, one or moreaspects of this gesture control process may be performed by a networkedserver (e.g., server 106) to take advantage of increased processingcapabilities and/or centralize the gesture input process.

For example, gesture control device 22 may receive field input data 64from one or more sensors 30. As described herein, sensors 30 may detectgestures from user 14 that are intended as gesture input (e.g.,indicator, control, termination, or confirmation gesture input). Gesturecontrol device 22 may be configured to connect to network 110 (e.g., awired or wireless network). In some examples, gesture control device 22may also be configured to communicate with networked server 106 vianetwork 110 to transmit field input data 64. Although network 110 may bea single network, network 110 may be representative of two or morenetworks configured to provide network access to server 106 and/orrepository 108. In general, gesture control device 22 may manage thecollection and/or generation of field input data 64 and request server106 to process field input data 64. However, in other examples, server106 may provide greater control over gesture control device 22, such asrequesting that gesture control device 22 begin obtaining field inputdata 64 from sensors 30.

Once received by gesture control device 22 via network 110, networkedserver 106 may be configured to store field input data 64 in repository108 until the field input data is to be analyzed and/or for long-termstorage. Both gesture control device 22 and networked server 106 mayconnect to network 110. Network 110 may be embodied as one or more ofthe Internet, a wireless network, a wired network, a cellular network,or a fiber optic network. In other words, network 110 may be any datacommunication protocol or protocols that facilitate data transferbetween two or more devices. Networked server 106 may also connect torepository 108 to store and/or retrieve field input data 64 receivedfrom gesture control device 22, gesture input types, gesture actions,gesture control schemes, or any other data stored in repository 108.

Networked server 106 and repository 108 may each include one or moreservers or databases, respectively. In this manner, networked server 106and repository 108 may be embodied as any hardware necessary to processand/or store field input data 64 and any other information related togesture input processing and control. Networked server 106 may includeone or more servers, desktop computers, mainframes, minicomputers, orother computing devices capable of executing computer instructions andstoring data. In some examples, functions attributable to networkedserver 106 herein may be attributed to respective different servers forrespective functions. Repository 108 may include one or more memories,repositories, hard disks, or any other data storage device. In someexamples, repository 108 may be included within networked server 106.

Repository 108 may be included in, or described as, cloud storage. Inother words, field input data 64 or any other such information may bestored in one or more locations in the cloud (e.g., one or morerepositories 108). Networked server 106 may access the cloud andretrieve the appropriate data as necessary. In some examples, repository108 may include Relational Database Management System (RDBMS) software.In one example, repository 108 may be a relational database and accessedusing a Structured Query Language (SQL) interface that is well known inthe art. Repository 108 may alternatively be stored on a separatenetworked computing device and accessed by networked server 106 througha network interface or system bus. Repository 108 may thus be an RDBMS,an Object Database Management System (ODBMS), Online AnalyticalProcessing (OLAP) database, or any other suitable data managementsystem.

Once server 106 receives field input data 64, server 106 may process thefield input data 64 to determine one or more gesture inputs from thefield input data and/or determine the corresponding control requested bythe gesture input. Server 106 may utilize any information or rulesstored in repository 108 and/or received from gesture control device 22along with field input data 64. For example, server 106 may generategesture control information 114 that specifies the action to becompleted by medical device 112 (e.g., IMD 18, a surgical device, animplantation device, or any other device configured to perform one ormore actions on patient 12 during, or as a result of, a medicalprocedure). Gesture control information 114 may be generated accordingto the techniques described herein to mitigate error that may occur withgesture input. Gesture control information 114 may include an adjustmentto a parameter value or some other command that at least partiallycontrols an action of medical device 112.

Server 106 may transmit gesture control information 114 to anotherdevice via network 110. For example, server 106 may transmit gesturecontrol information 114 to gesture control device 22 via network 110. Inresponse to receiving the gesture control information 114, gesturecontrol device 22 may transmit a control signal to medical device thatcommands medical device 112 to perform an action for a medical procedurerelated to patient 12. The control signal may be the same gesturecontrol information 114 or a signal derived from gesture controlinformation 114 and configured specifically for medical device 112. Inother examples, server 106 may transmit gesture control information 114to another device instead of gesture control device 22. Server 106 maytransmit gesture control information 114 to external programmer 50(e.g., a device that programs medical device 112) via network 110, andexternal programmer may respond by controlling medical device 112according to gesture control information 114. In some examples, server106 may transmit gesture control information 114 directly to medicaldevice 112 via network 110 when medical device 112 is connected tonetwork 110.

As described in FIG. 7, gesture input control of one or more medicaldevices may be performed by a distributed system over a network. In thismanner, gesture control device 22 may function together with server 106to control medical device 112 for a medical procedure. Server 106 may belocated remote from gesture control device 22 and/or user 14 (e.g., inanother building or even another city). Alternatively, server 106 may belocated within the same room as user 14 and still utilize network 110for transferring data. In some examples, multiple servers 106 may beaccessible by gesture control device 22 such that if one of the serversis not available for any reason, another server may be used to processfield input data 64. In other words, gesture control device 22 mayselect any available server 106 or a set of servers may operate togetherto provide redundancy for generating gesture control information 114.

FIGS. 8A and 8B are conceptual diagrams of example gestures for anindication gesture input and a control gesture input. As shown in FIG.8A, indication gesture input 120 includes a stationary index finger 122extended from hand 16A with the remaining fingers and thumb rolled intoa fist. Indication gesture input 120 may be used to signal the beginningof a gesture control mode such that gesture control device 22 can, forexample, enable the gesture control mode in response to detectingindication gesture input 120.

Indication gesture inputs, such as indication gesture input 120, mayresult from any different types of gestures. For example, an indicationgesture input may be a stationary configuration of one or two handsand/or a movement of one or two hands. In some examples, an indicationgesture input may include a stationary configuration of a hand (e.g.,hand 16A) to another hand (e.g., hand 16B) or anatomical structure suchas a chin, forehead, ear, or shoulder of user 14. Any of these or othergestures may be used and identified by gesture control device 22 as anindication control gesture per the stored instructions.

As shown in FIG. 8B, example control gesture input 124 includes hand 16Awith extended fingers 126 extended and positioned together. Controlgesture input 124 also includes movement of hand 16 in the direction ofone of arrows 128A or 128B. For example, movement of hand 16A in the“up” direction of arrow 128A may indicate an increase to a parametervalue for the medical device. Alternatively, movement of hand 16A in the“down” direction of arrow 128B may indication a decrease to theparameter value for the medical device. In some examples, the directionof hand 16A towards the direction of either arrow 128A or 128B may beidentified as separate gesture inputs.

A control gesture input, such as control gesture input 124, may includemovement of a hand between two locations in space. In other examples, acontrol gesture input may include repetitive hand motions (e.g.,circular movements, rotational movement such as representing the turningof a knob, up and down movements, or side to side movements). Othercontrol gesture inputs may include movement of two or more fingers withrespect to other fingers, movement of two hands with respect to eachother, sequential stationary hand configurations, extending a specificnumber of fingers.

Control gesture inputs may be different from indicator gesture input.For example, control gesture inputs may include some change of one ormore hands in space or configuration to indicate the desired change in aparameter value. An indicator gesture input may include a morestationary hand configuration that, when detected, causes gesturecontrol device 22 to enter the gesture control mode. Gesture controldevice 22 may present feedback to user 14 regarding the gesture, such asthe motion that was detected, the hand configuration detected, arepresentation of the actual gesture input detected, and/or theresulting change in parameter value identified based on the receivedgesture input. Indication gesture input may be received sequentially togesture control input. In other words, gesture control device 22 mayneed to detect the indication gesture input first such that the gesturecontrol mode can be entered, and gesture control device 22 can thendetect the control gesture input. Alternatively, indicator gesture inputmay be received simultaneously to the control gesture input (e.g., onehand provides the indicator gesture input while the other hand providesthe control gesture input).

FIGS. 9A and 9B are conceptual diagrams of example gestures for controlgesture input 130 and confirmation gesture input 134. As shown in FIG.9A, control gesture input 130 may include hand 16A with an extendedindex finger 122 with movement of hand 16A in the direction of arrow132A or 132B. In this manner, control gesture input 130 may provide amovement that indicates whether to increase or decrease a parametervalue. In other examples, two or more of the fingers of hand 16A may beextended during gesture control input 130. The number of fingers, or aspecific one or more fingers of hand 16A, may be extended during controlgesture input 130 to specify a respective parameter to be controlled bythe gesture input. Control gesture input 130 may be similar to controlgesture inputs described in FIG. 8B. In some examples, a control gesturemay require two hands instead of one.

Confirmation gesture input 134 may provide a relatively low errorgesture, i.e., a gesture that has a low likelihood of being erroneouslyinterpreted or recognized, to confirm that control gesture input 130 wasrequested. A low error gesture may include a hand configuration or handmotion that is easily identifiable and distinguishable from other typesof gestures. A low error gesture may also be a gesture that user 14would not typically make during ordinary actions of the user so that theconfirmation gesture input is not made inadvertently. For example,example confirmation gesture input 134 may include hand 16A with allfingers and thumb extended out and separated. Gesture control device 22may identify this configuration as the confirmation gesture input.

Confirmation gesture input 134 may be different from any of the gesturesthat may be used as a control gesture input. As discussed above, gesturecontrol device 22 may provide feedback to the user regarding whatgesture was detected. Control gesture input and confirmation gestureinput may be received sequentially such that the confirmation gestureinput is provided immediately after the control gesture input andwithout any intervening gesture. Alternatively, control gesture inputand confirmation gesture input may be provided simultaneously (e.g.,hand 16A may provide control gesture input 130 and hand 15B may provideconfirmation gesture input 134).

FIG. 10 is a conceptual diagram of example gesture that is authenticatedas provided by a particular user. Control gesture input 136 may be acontrol gesture input that is authenticated to a particular user beforegesture control device 22 controls an action based on the receivedcontrol gesture input. Control gesture input 136 may include hand 16Awith extended index finger 122 and hand 16B with index finger 138extended, and user 14 moves hands 16A and 16B closer or farther away,relative to one another, in the direction of arrow 140 to indicate thedesired change in control. For example, increasing the distance betweenfingers 122 and 138 may indicate an increase in the parameter value anddecreasing the distance may indicate a decrease in the parameter value.

Gesture control device 22 may analyze the gesture for one or moreaspects specific to the user 14. For example, gesture control device 22may measure a length of one or both of fingers 122 and 138, determine adifference in length between fingers 122 and 138, a rate of movement ofhands 16A and 16B, a positioning of a thumb, or any other aspects ofhands 16A and 16B or control gesture input 136 that may be specific to auser. Gesture control device 22 may require two or more aspects forauthentication in some examples. Gesture control device 22 may storecalibration data from previous gesture inputs of user 14 to determinethe gesture “fingerprint” specific to user 14. Authentication of controlgesture input 136 may prevent unauthorized or inadvertent gesture inputsprovided by someone other than the intended user 14.

FIGS. 11-17 are flow diagrams illustrating various processes andtechniques that may be used to detect and determine gesture input forcontrolling one or more actions of a medical procedure. Typically, thecontrol is of one or more medical devices that perform the action.Generally, one or more processors 60 of gesture control device 22 and/orprocessors 82 of networked server 106 are described as performing thedescribed processes in FIGS. 11-17. However, the processes may beperformed by one or more other devices or systems (e.g., externalprogrammer 50 or combinations of different processors and/or devices inother examples.) For example purposes, gesture control device 22 andprocessor 60 of system 10 will be described in the example techniques ofFIGS. 11-17. In other examples, a gesture detection module similar togesture detection module 88 of server 106 may perform these functions ofgesture control device 22.

FIG. 11 is a flow diagram that illustrates an example process forentering a gesture control mode in response to receiving an indicatorgesture input. As shown in FIG. 11, processor 60 of gesture controldevice 22 receives input field data from sensors 30 (150) and detectswhether or not an indicator gesture input was received from user 14(152). If processor 60 does not detect an indicator gesture input (“NO”branch of block 152), processor 60 continues to receive additional fieldinput data (150). If processor 60 does detect an indicator gesture input(“YES” branch of block 152), processor 60 enters the gesture controlmode that can accept control gesture input (154).

Processor 60 then receives additional input field data from sensors 30(156). If processor 60 does not detect a control gesture input (“NO”branch of block 162), processor 60 checks to determine if the gesturecontrol mode should be exited (158) (e.g., a termination gesture inputhas been received). If processor 60 determines that the gesture controlmode should not be exited (“NO” branch of block 158), processor 60 maycontinue to receive additional field input data (156). If processor 60determines that the gesture control mode should be exited (“YES” branchof block 158), processor 60 exits the gesture control mode (160) andcontinues to receive input field data (150).

If processor 60 does detect a control gesture input (“YES” branch ofblock 162), processor 60 determines the parameter value based on thedetected control gesture input (164). For example, processor 60 maysearch a database for parameter value that corresponds to the receivedcontrol gesture input or otherwise apply the control gesture input toone or more rules to generate the new parameter value. Processor 60 thencontrols a medical device to adjust the parameter to the new determinedparameter value (166). In this manner, processor 60 can control anaction for a medical procedure by controlling a medical device toperform the specified action requested by the gesture input. Processor60 then continues to receive input field data during the gesture controlmode (156) until the gesture control mode is exited.

FIG. 12 is a flow diagram that illustrates an example process forexiting a gesture control mode in response to a change in a medicalprocedure. As shown in FIG. 12, processor 60 of gesture control device22 monitors the status of a medical procedure such as which step iscurrently engaged by user 14 (170). Processor 60 analyzes the medicalprocedure and detect when a step of the medical procedure has beenstarted in which gesture control is allowed or required. If processor 60does not determine that the current step is associated with a gesturecontrol mode (“NO” branch of block 172), processor 60 continues tomonitor the medical procedure for changes (170). If processor 60 doesdetermine that the step of the medical procedure is associated with agesture control mode, (“YES” branch of block 172), processor 60 entersthe gesture control mode that can accept control gesture input (174).

Processor 60 then receives input field data from sensors 30 (176). Ifprocessor 60 does detect a control gesture input (“YES” branch of block178), processor 60 determines the parameter value based on the detectedcontrol gesture input and controls an action of a medical device thatcorresponds to the detected gesture input (180). For example, processor60 may control a medical device to adjust the parameter to a newlydetermined parameter value, based on the gesture input, during thecurrent step of the medical procedure. Although the step of the medicalprocedure may remain the same after completion of the action, completingthe action based on the control gesture input may trigger termination ofthe gesture control mode. Processor 60 then determines whether thestatus (or step) of the medical procedure has changed (182).

If processor 60 does not detect a control gesture input (“NO” branch ofblock 178), processor 60 checks to determine if the medical procedurestatus has changed (e.g., a new step has been entered or current stepexited) (182). If processor 60 determines that the status of the medicalprocedure has not changed (“NO” branch of block 182), the processorcontinues to receive input field data (176). If processor 60 determinesthat the medical procedure status has changed (“YES” branch of block182), processor 60 determines whether or not to exit the gesture controlmode (184) (e.g., a new step is not associated with the gesture controlmode). If processor 60 determines that the gesture control mode shouldnot be exited (“NO” branch of block 184), processor 60 may continue toreceive additional field input data (176). If processor 60 determinesthat the gesture control mode should be exited (“YES” branch of block184), processor 60 exits the gesture control mode (186) and continues toreceive input field data (170).

FIG. 13 is a flow diagram that illustrates an example process forreceiving a confirmation gesture input that confirms a control gestureinput. As shown in FIG. 13, processor 60 of gesture control device 22receives input field data from sensors 30 (190) and detects whether ornot a control gesture input was received from user 14 (192). Ifprocessor 60 does not detect a control gesture input (“NO” branch ofblock 192), processor 60 continues to receive additional field inputdata (190). If processor 60 does detect a control gesture input (“YES”branch of block 192), processor 60 determines whether or not aconfirmation gesture input was received (194).

If processor 60 does not detect a confirmation gesture input (“NO”branch of block 194), processor 60 ignores the detected control gestureinput (198). In other words, the confirmation control input is requestedbefore processor 60 will control any medical device based on the controlgesture input. In some examples, processor 60 may wait a predeterminedperiod of time or for some subsequent event (e.g., another controlgesture input) before determining that no confirmation gesture input wasdetected). If processor 60 determines that a confirmation gesture inputwas received (“YES” branch of block 194), processor 60 controls anaction of a medical device for the medical procedure corresponding tothe received control gesture input (196). For example, processor 60 maydetermine a new parameter value based on the detected control gestureinput and transmit the new parameter value to the medical device foradjustment of the parameter. Processor 60 then continues to receiveinput field data (190).

In the example of FIG. 13, processor 60 determines whether theconfirmation gesture input was received subsequent to the controlgesture input. However, in other examples, processor 60 may beconfigured to detect receiving the control gesture input simultaneouslywith the confirmation gesture input. In some examples, processor 60 mayeven require that the control gesture input was detected simultaneously(e.g., at least partially overlapping in time) with the confirmationgesture input. The confirmation gesture input of FIG. 13 may be used inconjunction with other error mitigation techniques, such as the gesturecontrol mode of FIGS. 11 and/or 12. In this manner, multiple errormitigation techniques may be used in conjunction with each other foradditional prevention of inadvertent gesture inputs.

FIG. 14 is a flow diagram that illustrates an example process forsuspending a gesture control mode in response to detecting an unexpectedobject within en envelope within which gesture input is detected. Asshown in FIG. 14, processor 60 receives input field data from sensors 30during an active gesture control mode (200). If processor 60 has notdetected an unexpected in the same envelope or area in which gesturesare detected by sensors 30 (“NO” branch of block 202), processor 60determines whether or not a control gesture input was detected (204). Ifprocessor 60 detects a control gesture input (“YES” branch of block204), processor 60 controls a medical device to perform an actioncorresponding to the control gesture input (206) and continues toreceive input field data (200). If processor 60 does not detect anycontrol gesture input (“NO” branch of block 204), processor 60 continuesto receive field input data from sensors 30 (200).

If processor 60 detects an unexpected object from the field input data(“YES” branch of block 202), the processor enters a safe mode that doesnot accept a control gesture input (208). In other words, processor 60does not act on any received control gesture input during the safe mode.The safe mode may also be referred to the period of time during whichthe gesture control mode has been suspended or exited. During the safemode, processor 60 continues to receive field input data from sensors 30(210). If processor 60 determines that the unexpected object remains inthe envelope (“YES” branch of block 212), processor 60 continues toreceive and analyze the field input data (210). If processor 60determines that the unexpected object no longer remains (“NO” branch ofblock 212), processor 60 exits the safe mode and re-enters (ofunsuspends) the gesture control mode such that processor 60 can againcontrol a medical device based on received control gesture input (214).

An unexpected object may be a third hand (indicating that another useris present and erroneous gestures may be detected) or an object that hasbeen placed between one or more of sensors 30 and user 14. In any case,processor 60 may determine that the unexpected object, or anomaly, mayreduce the accuracy of the field input data for gesture input detection.In some examples, processor 60 may end the safe mode in response todetermining other events. For example, processor 60 may end the safemode (i.e., re-enter the gesture control mode) in response todetermining that an error with input field data no longer exists, thesafe mode has reached a predetermined timeout, or determining that anon-gesture input over-riding the safe mode has been received.

FIG. 15 is a flow diagram that illustrates an example process forconfirming a gesture input is from an expected user. As shown in FIG.15, processor 60 determines the user of gesture control device 22 or theuser that should otherwise be providing gesture input (220). This stepmay require receiving an input identifying the user (e.g., usernameand/or password or biometric parameter). Once the user of the system hasbeen determined, processor 60 of gesture control device 22 receivesinput field data from sensors 30 (222). Processor 60 then detectswhether or not a control gesture input was received (224). If processor60 does not detect a control gesture input (“NO” branch of block 224),processor 60 continues to receive additional field input data (222).

If processor 60 does detect a control gesture input (“YES” branch ofblock 224), processor 60 compares the control gesture input to one ormore attributes of a stored gesture type for the identified user (226).In other words, processor 60 may determine that one or more aspects ofthe received control gesture input correlates to respective attributesof the user. Aspects of the received control gesture input may includeone or more dimensions of the hands and/or fingers in the controlgesture, a rate of the motion of the gesture, a magnitude of the controlgesture, a position within the envelope of sensors 30 for which thegesture was received, or any other aspect. The attributes of the usermay be stored based on a calibration session in which the userpreviously performed the control gesture input or other measurements ormotions regarding fingers, hands, or any other anatomical structure.

If processor 60 cannot confirm that the control gesture input correlatesto the expected user (“NO” branch of block 228), processor 60 may ignorethe detected control gesture input (230) and continue receiving fieldinput data (222). In other examples, processor 60 may suspend or exitthe gesture control mode in response to determining that the controlgesture input does not correlate to the expected user. If processor 60confirms that the control gesture input corresponds to the expected user(“YES” branch of block 228), processor 60 controls an action of amedical device for the medical procedure corresponding to the receivedcontrol gesture input (232). Processor 60 then continues to receiveinput field data (222).

If the user determines that processor 60 is not correctly identifyingthe user, processor 60 may be configured to receive user inputrequesting recalibration of gesture input. Processor 60 may then confirmthe identity of the user and perform a calibration application for theuser in which attributes of the user can be derived from examplegestures performed by the user. In response to completion of thecalibration process, processor 60 may again enter the gesture controlmode for the user and again receive input field data (222).

FIG. 16 is a flow diagram that illustrates an example process fordetecting a gesture input based on data generated from an externalsensor and an IMD. In this manner, the IMD may provide data thatconfirms the gesture input is accurate. As shown in FIG. 16, processor60 of gesture control device 22 receives input field data from sensors30 (240). Processor 60 then detects whether or not a control gestureinput was received (242). If processor 60 does not detect a controlgesture input (“NO” branch of block 242), processor 60 continues toreceive additional input field data (240).

If processor 60 does detect a control gesture input (“YES” branch ofblock 242), processor 60 receives external control data from a sensor ofIMD 18, for example (244). The IMD sensor may be one or more electrodes,one or more accelerometers, one or more ultrasound transducers, or anyother sensors that may detect the presence of the hands of user 14.Processor 60 may then compare the control gesture input to the externalcontrol data from IMD 18. For example, processor 60 may determine thatthe control gesture input occurred within a same area in which IMD 18detected the presence of the hands. Alternatively, IMD 18 may detectmotion of the detected hands, instead of just indicating that the handswere detected, in order to provide additional information regarding themotion of detected hands with respect to IMD 18. Processor 60 may thencorrelate the gesture control input to the general motion detected byIMD 18. In any event, processor 60 may use the comparison of the gesturecontrol input to the data from IMD 18 to confirm the accuracy of thecontrol gesture input.

If processor 60 cannot confirm that the control gesture input correlatesto the data from IMD 18 (“NO” branch of block 248), processor 60 mayignore the detected control gesture input (250) and continue receivingfield input data (240). In other examples, processor 60 may suspend orexit the gesture control mode in response to determining that thecontrol gesture input does not correlate to the expected user. Ifprocessor 60 confirms that the control gesture input is accurate (“YES”branch of block 248), processor 60 controls an action of a medicaldevice for the medical procedure corresponding to the received controlgesture input (252). Processor 60 then continues to receive input fielddata (240).

In other examples, processor 60 may receive the external control datacontemporaneously (e.g., at the same time) as the field input data. Inthis manner, processor 60 may periodically or continuously receive bothfield input data from sensors 30 and external control data from IMD 18.Alternatively, processor 60 may only request external control data fromIMD 18 in response to detecting control gesture input. Although externalcontrol data has been described for use in confirming control gestureinput, processor 60 may confirm other gesture inputs, such as indicatorgesture inputs, confirmation gesture inputs, or termination gestureinputs, in other examples.

FIG. 17 is a flow diagram that illustrates an example process forlimiting adjustments to parameter values for gesture inputs. As shown inFIG. 17, processor 60 of gesture control device 22 receives user inputrequesting adjustment of a parameter value (260). Processor 60 thendetermines whether or not the user input was a gesture input or an inputreceived via non-gesture input devices (262). If processor 60 determinesthat the user input is not a gesture input (“NO” branch of block 262),processor 60 applies a default rate of change factor and defaultadjustment range factor for the user input controlling the medicaldevice (264). The default rate of change factor or default adjustmentrange factor may be a factor applied to signals from knobs, switches, orsliding devices. The rate of change factor indicates how much to adjusta parameter value for a corresponding movement of the input device. Thedefault adjustment range factor indicates the amount of change that canoccur to a parameter value from a single movement of the input device.Processor 60 may then adjust the parameter value based on the user inputand the default factors (266) and again receive user input (260).

If processor 60 detects the user input is a gesture input (“YES” branchof block 262), processor 60 determines the type of gesture input (268).The type of gesture input may be the specific gesture that correspondsto a respective action of a medical device. For example, the type ofgesture input may be a finger pinch or a hand elevation change.Processor 60 then applies a limited rate of change factor and/or alimited adjustment range factor associated with the type of gestureinput received (270). The limited rate of change factor may limit howfast a parameter value can be changed due to hand motion (i.e., preventlarge magnitude parameter value changes due to small hand movements).The limited adjustment range factor may limit the magnitude of parametervalue adjustment from a single gesture input. These limited factors mayprevent inadvertent or unintended parameter value changes due to gestureinput. Processor 60 may then adjust the parameter value based on thegesture input and the one or more limited adjustment factors (272).Processor 60 may then again receive user input (260).

The limited rate of change factor and/or limited adjustment range factormay be specific to the type of gesture input. For example, the limitedrate of change factor for a single hand movement gesture may be greaterthan the limited rate of change factor for a gesture of two hands movingwith respect to each other (e.g., the single hand movement input is morelimited to the adjustment that can occur). Each gesture type, or groupof gesture types, may have respective limited factors. The limitedfactors may be based on calibration from user provided gestures and/orthe user may directly adjust the limited factors. In some examples, thelimited factors may be different for different steps of the medicalprocedure.

The disclosure contemplates computer-readable storage media comprisinginstructions to cause a processor to perform any of the functions andtechniques described herein. The computer-readable storage media maytake the example form of any volatile, non-volatile, magnetic, optical,or electrical media, such as a RAM, ROM, NVRAM, EEPROM, or flash memorythat is tangible. The computer-readable storage media may be referred toas non-transitory. A programmer, such as patient programmer or clinicianprogrammer, or other computing device may also contain a more portableremovable memory type to enable easy data transfer or offline dataanalysis.

The techniques described in this disclosure, including those attributedto gesture control device 22, external programmer 50, networked server106, or medical devices 18 or 112, and various constituent components,may be implemented, at least in part, in hardware, software, firmware orany combination thereof. For example, various aspects of the techniquesmay be implemented within one or more processors, including one or moremicroprocessors, DSPs, ASICs, FPGAs, or any other equivalent integratedor discrete logic circuitry, as well as any combinations of suchcomponents, embodied in programmers, such as clinician or patientprogrammers, stimulators, remote servers, or other devices. The term“processor” or “processing circuitry” may generally refer to any of theforegoing logic circuitry, alone or in combination with other logiccircuitry, or any other equivalent circuitry.

Such hardware, software, firmware may be implemented within the samedevice or within separate devices to support the various operations andfunctions described in this disclosure. For example, any of thetechniques or processes described herein may be performed within onedevice or at least partially distributed amongst two or more devices,such as between gesture control device 22 and networked server 106. Inaddition, any of the described units, modules or components may beimplemented together or separately as discrete but interoperable logicdevices. Depiction of different features as modules or units is intendedto highlight different functional aspects and does not necessarily implythat such modules or units must be realized by separate hardware orsoftware components. Rather, functionality associated with one or moremodules or units may be performed by separate hardware or softwarecomponents, or integrated within common or separate hardware or softwarecomponents.

The techniques described in this disclosure may also be embodied orencoded in an article of manufacture including a computer-readablestorage medium encoded with instructions. Instructions embedded orencoded in an article of manufacture including a computer-readablestorage medium encoded, may cause one or more programmable processors,or other processors, to implement one or more of the techniquesdescribed herein, such as when instructions included or encoded in thecomputer-readable storage medium are executed by the one or moreprocessors. Example computer-readable storage media may include randomaccess memory (RAM), read only memory (ROM), programmable read onlymemory (PROM), erasable programmable read only memory (EPROM),electronically erasable programmable read only memory (EEPROM), flashmemory, a hard disk, a compact disc ROM (CD-ROM), a floppy disk, acassette, magnetic media, optical media, or any other computer readablestorage devices or tangible computer readable media. Thecomputer-readable storage medium may also be referred to as one or morestorage devices.

In some examples, a computer-readable storage medium comprisesnon-transitory medium. The term “non-transitory” may indicate that thestorage medium is not embodied in a carrier wave or a propagated signal.In certain examples, a non-transitory storage medium may store data thatcan, over time, change (e.g., in RAM or cache).

Various examples have been described herein. Any combination of thedescribed operations or functions is contemplated. These and otherexamples are within the scope of the following claims.

What is claimed is:
 1. A method comprising: receiving, by a processor,an indication to enter a gesture control mode during which the processoris configured to control one or more actions related to a medicalprocedure; responsive to receiving the indication, entering, by theprocessor, the gesture control mode; monitoring, by the processor, fieldinput data received during the gesture control mode for one or morecontrol gesture inputs, the one or more control gesture inputsrequesting the processor control the one or more actions related to themedical procedure; determining, by the processor and during the gesturecontrol mode, to exit the gesture control mode; and responsive todetermining to exit the gesture control mode, exiting, by the processor,the gesture control mode, wherein exiting the gesture control modedisables gesture input control of the one or more actions related to themedical procedure.
 2. The method of claim 1, wherein determining to exitthe gesture control mode comprises: monitoring progress of the medicalprocedure, wherein the medical procedure comprises a plurality of steps,one step of the plurality of steps being configured to accept the one ormore control gesture input controlling the one or more actions relatedto the medical procedure during the one step; determining that the onestep of the medical procedure has been completed; and responsive todetermining that the one step has been completed, exiting the gesturecontrol mode.
 3. The method of claim 2, wherein the one step is a firststep of the plurality of steps, and wherein the method furthercomprises: accepting the one or more control gesture inputs of a firstplurality of gesture inputs during the first step; determining that themedical procedure has entered a second step of the plurality of steps ofthe medical procedure; responsive to the determination that the medicalprocedure has entered the second step of the plurality of steps of themedical procedure, entering the gesture control mode; accepting one ormore gesture inputs of a second plurality of gesture inputs during thesecond step, the second plurality of gesture inputs being different thanthe first plurality of gesture inputs; determining that the second stepof the medical procedure has been completed; and responsive todetermining that the second step has been completed, exiting the gesturecontrol mode for the second step.
 4. The method of claim 1, whereindetermining to exit the gesture control mode comprises: tracking aduration of time from when the gesture control mode was entered;comparing the duration of time to a predetermined gesture controltimeout; determining that the duration of time exceeds the predeterminedgesture control timeout; and responsive to determining that the durationof time exceeds the predetermined gesture control timeout, exiting thegesture control mode.
 5. The method of claim 4, wherein thepredetermined gesture control timeout is a first gesture control timeoutspecific to a first step of a plurality of steps of the medicalprocedure, the gesture control mode is entered during the first step,and the first gesture control timeout is different from a second gesturecontrol timeout specific to a second step of the plurality of steps ofthe medical procedure.
 6. The method of claim 1, further comprising:receiving first field input data during the gesture control mode;detecting, from the first field input data, an unexpected object in anenvelope within which gesture input can be sensed; responsive todetecting the unexpected object, suspending the gesture control mode toprevent detection of erroneous gesture input due to the unexpectedobject; receiving second field input data during the suspension of thegesture control mode; detecting, from the second field input data, thatthe unexpected object is no longer in the envelope; and responsive todetecting that the unexpected object is no longer in the envelope,re-entering the gesture control mode.
 7. The method of claim 1, furthercomprising: receiving, during the gesture control mode, an indication ofone or more control gesture inputs requesting the processor control theone or more actions related to the medical procedure; receiving, duringthe gesture control mode, a confirmation gesture input confirming theone or more control gesture inputs; and responsive to receiving both theone or more control gesture inputs and the confirmation gestureconfirmation input, controlling, based on the one or more controlgesture inputs, the one or more actions related to the medicalprocedure.
 8. The method of claim 1, wherein control of the one or moreactions related to the medical procedure comprises adjustment of aparameter that at least partially defines a medical therapy.
 9. Themethod of claim 1, wherein control of the one or more actions of themedical procedure comprises control of a surgical device to operate withrespect to a patient receiving the medical procedure.
 10. The method ofclaim 1, wherein the medical procedure comprises a surgical procedure.11. The method of claim 1, wherein the medical procedure comprises animplantation of one or more medical devices.
 12. The method of claim 1,wherein the medical procedure comprises adjusting one or more parametersthat at least partially define electrical stimulation therapy.
 13. Themethod of claim 1, wherein the processor is housed within one of anetworked server, a medical device programmer, or an implantable medicaldevice.
 14. A system comprising: one or more processors configured to:receive an indication to enter a gesture control mode during which theprocessor is configured to control one or more actions related to amedical procedure; responsive to receiving the indication, enter thegesture control mode; monitor field input data received during thegesture control mode for one or more control gesture inputs, the one ormore control gesture inputs requesting the processor control the one ormore actions related to the medical procedure; determine, during thegesture control mode, to exit the gesture control mode; and responsiveto determining to exit the gesture control mode, exit the gesturecontrol mode, wherein exiting the gesture control mode disables gestureinput control of the one or more actions related to the medicalprocedure.
 15. The system of claim 14, wherein the one or moreprocessors are configured to determine to exit the gesture control modeby: monitoring a progress of the medical procedure, wherein the medicalprocedure comprises a plurality of steps, one step of the plurality ofsteps being configured to accept the one or more control gesture inputscontrolling the one or more actions related to the medical procedureduring the one step; determining that the one step of the medicalprocedure has been completed; and responsive to determining that the onestep has been completed, exiting the gesture control mode.
 16. Thesystem of claim 15, wherein the one step is a first step of theplurality of steps, and wherein the one or more processors areconfigured to: accept the one or more control gesture inputs of a firstplurality of gesture inputs during the first step; determine that themedical procedure has entered a second step of the plurality of steps ofthe medical procedure; responsive to the determination that the medicalprocedure has entered the second step of the plurality of steps of themedical procedure, enter the gesture control mode; accept one or moregesture inputs of a second plurality of gesture inputs during the secondstep, the second plurality of gesture inputs being different than thefirst plurality of gesture inputs; determine that the second step of themedical procedure has been completed; and responsive to determining thatthe second step has been completed, exit the gesture control mode forthe second step.
 17. The system of claim 14, wherein the one or moreprocessors are configured to determine to exit the gesture control modeby: tracking a duration of time from when the gesture control mode wasentered; comparing the duration of time to a predetermined gesturecontrol timeout; determining that the duration of time exceeds thepredetermined gesture control timeout; and responsive to determiningthat the duration of time exceeds the predetermined gesture controltimeout, exiting the gesture control mode.
 18. The system of claim 14,wherein the one or more processors are configured to: receive firstfield input data during the gesture control mode; detect, from the firstfield input data, an unexpected object in an envelope within whichgesture input can be sensed; responsive to detecting the unexpectedobject, suspend the gesture control mode to prevent detection oferroneous gesture input due to the unexpected object; receive secondfield input data during the suspension of the gesture control mode;detect, from the second field input data, that the unexpected object isno longer in the envelope; and responsive to detecting that theunexpected object is no longer in the envelope, re-enter the gesturecontrol mode.
 19. The system of claim 14, wherein control of the one ormore actions related to the medical procedure comprises adjustment of aparameter that at least partially defines a medical therapy.
 20. Thesystem of claim 14, wherein control the one or more actions of themedical procedure comprises control of a surgical device to operate withrespect to a patient receiving the medical procedure.